First of all you mentioned some stuff like trading, Traveling Merchants, or the wholesaler. Don't look there because none of that will work. Granite mines are your only possible source of granite. And as mentioned upthread, you can increase the effectiveness of your existing granite mines both by upgrading them and by upgrading the portal. But finally, I don't think 3 granite mines is nearly enough. I think when I did Dwarves I had 13 granite mines and 5 copper foundries. Mind you, that was for a very fast pace through the chapter. If you wanted to take your time in the chapter you could get by with fewer.
One thing that can help you make room for your settlement in Dwarves is upgrading your residences. When you do the Dwarf level upgrades to your houses, they actually become slightly smaller. Elvish players will save 1 space per residence and human players will save 2 spaces per residence. They also become a narrow rectangle, so if you orient them with the skinny end facing your roads you are saving 1 square of road frontage per residence. It does not sound like much on an individual basis, but when you upgrade all your residences it really adds up.