Early chapters, I pretty consistently deleted my culture buildings and rebuilt with more current ones (eg, delete Flying Boats when I unlocked Spot of Whispering Trees). I also played frequently though, so I consistently had supplies to spare.
Once I started getting through major events and had better buildings, I deleted less frequently.
Also, not the question you directly asked, but its a good idea to "expand" (meaning choose expansions) so that your city is a square. It makes it much easier to reorganize as your city grows.