If there are ways to manipulate or rig the system to an outcome which suits an attackers goals, then it's no good. We can't inadvertently introduce that kind of weakness.
There will always be a way to manipulate the system even if we increase the block size up to 1 TB. Miners have a huge advantage, they can send transactions with very high fees while losing nothing because all the paid fees finally end up in their pocket. So, if I am a big miner and make tens of thousands of transactions and pay millions in fees, I will increase fee for everyone and collect additional money from it but I also get all my spent millions back because fees and block reward comes together to miners.
On other hand, miner/pool also have disadvantage where their block took longer to be verified and propagated. In case their mined block become stale due to their manipulation attempt, they'd lose some of their income.