You don't have to upgrade it, you can keep it at the present level.
Since you can sell off the mercenary camp and the traininggrounds it probably was set up to make sure everyone had at least some possibility to create troops as there are requirements for those in quests, whether you use them or not.