The fear is that a cartel of big, centralized, have-huge-data-pipes miners would drive out smaller miners by forcing up the block size high enough so the smaller miners have to drop out.
Sounds like an irrational fear to me. Big centralized miners couldn't care less about little miners. Little miners are insignificant. And what are these big centralized miners going to use to fill up these huge blocks? Fake transactions? That makes no sense. Real transactions? Then there's a real need for such huge block sizes.
Yes they can make fake transactions to fill blocks. That would partially shut some miners with small bandwidth. The occasional very large blocks might cause the smaller blocks to not propagate and thus get orphaned. The low bandwidth miners would need to make up for it by increasing their hashrate if they have other advantages like cheap power or government subsidies. It's just another competition vector.