The spire and the academy work hand in hand..... You can trade in all of those spell fragments for temporary military buildings, timers, relics, KP, pet food, ancient knowledge, portal profits, merchant buildings, top of the line per square culture buildings, ECT.
Maybe it's because I have a level five magic academy that keeps CC's turning 24 hours a day but I cannot keep up with spell fragments. I am still kind of new, I've been playing the spire every week since March and disenchant all of my event buildings and my small "stockpile" took a big hit crafting the three bears and upgrading the artifacts to last years. I am looking forward to being in the situation where crafting all of those wonderful things in the magic academy just is not enough to keep me happy anymore ; - )
Edited to add: I just read what I wrote and it came off as kind of smart ass.....that was not my intention I really am looking forward to being in that situation.