Even if miners only can ignore transactions, they will do so - including more transactions means slower block propagation times and as already said in a different thread, bandwidth is still a limiting factor in an unlimited/virtually unlimited block size scenario.
The thing is that hashing adds directly to proof of work. A miner's costs become proportional to his hash power. There is an economy of scale here. If the optimal point is at 0.01% of the total hashing power, then things stay distributed. If economies of scale continue so that a miner with 90% of the hash power is more efficient than one with 10% of the hashing power, then you end up with a "natural" monopoly.
A situation where things might end up as a natural monopoly must be avoided.
If miners become bandwidth limited, then an entirely new dynamic takes over.
This means that the primary cost for a miner is bandwidth. In this situation, a miner's costs are not proportional to his hashing power. A miner with 90% of the market only needs to pay for bandwidth once and gets 90% of the fees, while a miner with 10% of the hashing power also pays the same, but only gets 10% of the fees. This puts the 10% guy out of business.
Bandwidth usage doesn't add to the formal proof of work of the main chain, so it is just a deadweight loss, without adding security.
If most of a miner's costs are hashing power, then things are much less likely to end up in the natural monopoly situation (unless hashing ends up with massive economies of scale too).