That's a standard rule not a consensus rule. Technically your output script with or without OP_RETURN can be so big so that it fills the entire block. In other words there is no limit on its size other than block size.
why would a scriptpubkey need to be unlimited in size for what is supposed to be a "payment system"? you have to define a strict format for a transaction call it a transaction template and anything that doesn't matchup to that template gets rejected. for a payment system, you only need a template like this:
sending address: receiving address: amount
bitcoin really made a mess of things with its utxo model and using SCRIPT language and all these different address types. but we're stuck with it. no going back. the only question is "can bitcoin survive as a payment system?"
now, there is no such thing as utxos anymore. it will be account based. that way there's no extra overhead for people sending things to burn addresses other than adjusting its balance.
Some of your question/thought could be answered by the fact satoshi wanted to support all kinds of transaction.
The nature of Bitcoin is such that once version 0.1 was released, the core design was set in stone for the rest of its lifetime. Because of that, I wanted to design it to support every possible transaction type I could think of. The problem was, each thing required special support code and data fields whether it was used or not, and only covered one special case at a time. It would have been an explosion of special cases. The solution was script, which generalizes the problem so transacting parties can describe their transaction as a predicate that the node network evaluates. The nodes only need to understand the transaction to the extent of evaluating whether the sender's conditions are met.
--snip--
--snip--
And FYI, you could add script directly on scriptpubkey, rather than using P2SH, P2WSH or P2TR. Such transaction which do that would be considered as non-standard though.