If the block size is pushed upward, then fewer individuals can afford to mine, and there are centralization risks with mining hardware ownership.
If you push the block size upward, there is little change compared to today - individual miners still choose what pool they direct their ASICs to. It cant be any worse.
On the other side with artificially keeping small blocks, unable to use onchain Bitcoin transactions is definitively increased centralization, as users have to use centralized offchain solutions (similar how bank or paypal works).