You can't get rid of the Barracks once it is built. The best you could do is not upgrade it any more.
But I still find those troops useful. And I am in Chapter 16, for reference. I am not a meticulous fighter, though, so if fighting is your absolute favorite part of the game, they may not be as useful.