economic/social complexity - people signed up to Bitcoin on the premise that the value of bitcoins are maintained by their scarcity, with a specific inflation schedule. Changing that will destroy the Schelling point and nobody will agree to it. On the other hand, nobody signed up to Bitcoin on the premise of a specific block size limit,
The scarcity of bitcoin's can not be maintained without security, the worth of Bitcoin as a transaction medium can't be maintained without security. The comparitive benefits of Bitcoin over alternatives depend on decenteralization. Security and decentralization are not possible without block size scarcity.
I certainly signed up for limited size blocks and so did Satoshi, as the limit is baked into the code not just a soft limit as many limits are (including the 500k maximum block target) but a hard network rule. Absent this rule Satoshi's economic argument that fees will eventually provide the incentive for honest participation once the subsidy has become small is invalid. There _must_ be competition for block space to make fees a viable way to pay for security.
Without a limit which is reasonably within the bounds of inexpensive hardware there will be great pressure to let someone else handle the validation, and you end up with an outcome similar to the central banks in democratic countries. Bitcoin would be just another very technically inefficient way to have a trust laden monetary system.
So you will have this beautiful decentralized system that everyone can run but nobody can use, they will have to resort to 3rd party alternatives with various trust requirements.
The overwhelming majority of gold and USD transactions are not done with gold bars or US bills can no one use these currencies.
Your use of 'Various' there is deceptive. Various means people have a choice. It's perfectly possible to build fully decentralized payment system denominated in Bitcoin. But if Bitcoin itself is bloated then these options can't exist because they they would have both their own high cost of operation plus the unimaginably high cost of operation of a bitcoin shipping around gigabyte blocks.
It's more likely the block limit will be raised eventually to about 100 MB. People won't be able to run a full node on their cellphones, but any enthusiast will still be able to do it on a PC, and it will allow for 1% of payments to take place on the blockchain. It's still sufficiently decentralized and is now also useful.
And on this point we do not completely disagree, though I think 100MB is rather high. I'm not saying that adjusting the limit is a complete non-starter. Only that removing it or setting it to some value which is not utterly uncontroversial long after most practical devices can handle it is a non-starter. My point in responding here is that arguments that it will soon be harder to handle and only something that major businesses can support are bogus. It may or may not be increased in the future, but if it is it will only happen to a level which is uncontroversial.