I will second what Lalanya said. Both your residences and your armories will grow drastically in size as well as change shape. Do these upgrades early in the chapter, before building your settlement. This will give you plenty of room to shuffle things around and rearrange your streets.
Your Fairies level residences are 2x4 in size. When you do the Orc level upgrades to them they will become 4x3. Therefore, they become both bigger and the direction of the rectangle changes orientation. The good news is that they will also provide a huge increase in population when upgraded. You will likely be able to reduce your total number of residences and still gain population when you upgrade them.
Your armories are currently 6x3 in size. They will become a 5x5 square when upgraded. It is very important to do the armory upgrades in the Orcs chapter early, because when you upgrade them they gain the Breeding Pit ability. This means that in addition to boosting your training size, armories will produce orcs. In fact, your armories will be your primary source of orcs. You can't do the chapter without a good supply of orcs.