Upgrading the portal will increase the production in all of your settlement buildings. I would focus on your portal, then copper, then granite. If you have anymore space, I would try to add 4 more granite and 2 more copper. Oh and for me, I never would upgrade coppers past level 3, the amount of granite was not justifiable and would have kept me in the chapter for longer.