Nobody has responded to the most relevant part:
Previous periods of full blocks didn't have as many people who NEEDED not just wanted the network to work. The more people who NEED it to work, the more vulnerable it is to a spam attack. The blocksize limit is an effective way to not just keep bitcoin small, but to keep it unimportant.
only about 40% of current TX "need" the network to work. There's plenty of fat to chop away and the spam goes down and gets more costly as fees go up, gosh darn, think about that!
You are not differentiating small legitimate transactions from hostile transactions. What I am saying is that when you get rid of the small legit xactions, the chain still fiils up with large legit transactions as bitcoin grows until it becomes vulnerable to a hostile attack. Those small legit transactions are acting as a buffer and you want to get rid of them.
... there is no way to differentiate between "small legit transactions" and fee-paying spam if they pay the same fee, so no, there is no 'buffer' as you are imagining.
And you are thinking all upside-down. Nobody wants "to get rid of" TX ... for the last 6 years free transactions (and yes, abundant spam) have been tolerated and given gold-plated security that only the bitcoin blockchain can provide. At some point, the network becomes saturated because the demand for gold-plated bitcoin TX will for the foreseeable future overcome the network's technological constraints and ability to supply it ... so now the question is "what price?", hence fees.
The block limit is the technological limit that constrains the on-chain capacity, the arise of fees is the free market reaction.