Since the majority of nodes in the beginning ran the bitcoin core, that core became the spec, and any other implementation that deviated from the core would be kicked out of the system.
I think the best course of action would be to acknowledge that the prototype has served its purpose and retire it in favour of treating a non-prototype codebase as the reference implementation for which a specification can be written.
Even if we did this, and put a large amount of open effort into creating the a new perfect core, we would still have the above issue. That core would be the specification if the majority ran it, including all of it's unknown bugs.
What we should aim for is at least three full node implementations in the wild, with no single implementation having majority usage.
Also, a clear spec with extensive testing of each implementation's correctness.
As soon as we make a transition away from a monoculture (one implementation has a majority), we'll have an upper bound on the severity of likely chain forks. If 1 of n implementations disagrees with the other (n-1), then it's clearly a bug in that particular implementation that should be fixed.
The worst case scenario is a bug that causes more than one implementation to diverge at the same time, but that's no worse than what we've already been through in the past and the odds of that happening can be continually reduced with a good QA program.
That sounds like an ideal plan. Not sure how to put it into action though. The current blocksize debate if anything shows how the current devs have conflict of interests in even considering this as a direction.
One option is to start with a move towards adoption of multiple implementations before making the "spec". If Bitcoin XT and btcd both gained enough adoption vs. the Satoshi client that all three were below 50% of the network, then we would now have a situation where a bug in the Satoshi client was no longer the spec, and the Satoshi client would have to quickly bug fix itself to adhear to the XT and btcd majority (and same for XT and btcd).
This situation would then force the drafting of a real specification. With one dominate implementation the need for a spec is diminished. But with several implementations the need for them to draft what is agreed becomes quite strong.