The narrative pivot just happened to fit well with their overall rebellion against established shounen tropes and archetypes. It’s what makes it so enthralling. Something they have thankfully kept in the remakes. It’s a shounen only superficially, it has all the flashy action bits and colorful high tech things, the blank MC, and the harem cast. But then you get to the characterization and dialogue and it’s pure existentialism horror.
It’s not that deep. They got the rug pulled from under them when the studio slashed the budget. Instead of doing an ending they wanted, lots of action, with bad animation, they instead rebelled with a rewrite ending that used intentional vagueness and ambivalent philosophical storytelling that let them use their resources to focus on art design and keep the cool factor up. Tons of stills, but really pretty. That’s why a lot of the last sequences are just artsy backgrounds with only voice overs, there are several seconds of just white paper, etc. This however proved widely popular and the writing was vague but interesting enough to fuel hundreds of hours of video analysis and thousands of words essays.
The other factor was the mythological treatment of biblical imagery, which was rather new and unique at the time.
I’ve read it both ways with sources. Read also a third that they never had the whole budget to begin with and overextended due to clerical error. Alledgelly the latest remake has the style of ending they originally envisioned with lots of fight sequences and more experimental animation. It’s part of the endless debate.