Having fewer peers =! being centralized.
What's the point of keeping bitcoin crippled for a niche market? There are plenty of altcoins for this exact purpose. Bitcoin's destiny is world domination, for anything else there is an altcoin and everyone is happy.
We should be careful with that one.
You need to ask yourself then "Who is going to dominate Bitcoin?"
Would that be a "big block institution" or a simple and robust framework of transparent monetary rules enforced by the majority of Bitcoin users?
The latter seems more in line with Bitcoin's true purpose.