In my first FS, we only got 5 chests in tourney and if we wanted anymore, I had to do more of the heavy lifting myself. As they say, necessity is the mother of invention! I had to get really good at stretching my army out to do more. That's how I started learning the nuances of manual fighting. Eventually, I was able to do more than the AM, who was in late game, despite my stick figure troops. This was also before they put buildings from previous events in the Spire/MA. Since I wasn't here for the bear/phoenix events, no Brown Bear or Fire Phoenix for me to play with. Once I knew I can do just as well or better with level 1 and 2 troops than someone with all the toys, I knew fighting was only going to get easier once you give me all those toys later in the game. So don't get discouraged. It does get easier!
The ratio of your troop to enemy troop is set for everyone at the same point in the Spire, whether you're in chap 5 or 21, and however which way you've decided to build your city. Therefore, as you acquire more toys, you are really fighting the same ratio, but now with more oomph behind your troops. Hence, it can only get easier. As your base squad size increases, it's more about being able to make back the dead troops in time to go again next week. That is a troop production issue more than a strength issue. The main difference between Spire and tourney is the multi-wave nature of the later encounters. Therefore, you do need to have enough health to survive, even if you can kill rather quickly. Monastery/Sanctuary will help with this, or dropping DAs/UUUs. If you get your military wonders high enough though, you can fight the Spire without Fire Phoenix or buffs. Until then, it's probably advised to fight the Spire with a fed Fire Phoenix and some buffing.