If you mean the buildings from the other guest races (dwarves, fairies), you won't need them at all after you finish the chapter they were in. Any goods you have, you will keep, but you won't have a way to know how much you have. The only use for those goods would be to build more of that guest race's roads. I'm just now in fairies. There was a quest to sell the dwarven portal, and shortly after I did that, I sold off all the portal tracks, granite mines, and copper foundries. I used that space to start my fairy settlement. When I leave fairies, I'll do the same thing going into Orcs. From what I've heard, you need a bunch more space for Orcs than for the other 2 settlements, but I haven't looked closely at the different ways to do it yet.