Up to that point if a majority of the miners downgraded (or never actually upgraded - just signalled) it would not manage to actually maintain the longest chain so if someone published a currently valid tx that was invalid under taproot we would have a chain split... (I think I am right on this?)
The purpose of the delayed activation is to give plenty of time for everyone to upgrade.
Bitcoin doesn't follow the longest chain, it follows the longest valid chain-- and at least to upgraded parties any taproot-invalid chain wouldn't be valid. So the idea is that it doesn't even matter too much if such an invalid chain is created, as it'll just get ignored and the effect would just be some hashrate vanishing (until the losses cause them to wake up and fix their stuff

). And this holds no matter how much hashrate there is...
Of course, parties who aren't transacting and are asleep at the switch might not have upgraded but if they're not accepting transactions in realtime it doesn't much matter what chain they are on.

So I think that really takes the wind out of the chaos conspiracy theory.

But it's a good reason to get major exchanges and other influential parties that accept a lot of payments to confirm that they've upgraded prior to activation.
Also, "someone published" isn't sufficient. Pre-taproot nodes (and miners) will not relay or mine taproot spends. So the only way a block could be created with an invalid spend in it is if some miner intentionally did so (or at least lobotomized the protection out of their node), even if their software wasn't upgraded.