I don't think it has anything to do with being a "rich guy's club." Being able to run a node shouldn't be a "rich guy's club" either. And though mining already is, do we want to exacerbate that? Maxblocksize is very much about preventing the externalities that limit node/miner participation into more and more centralized groups.
I think "artificial limit" implies a misconception. Bitcoin as a system was a matter of designing a system that optimally aligns market incentives to ensure a decentralized, secure network. We cannot have talk about an "artificial limit" without mention that this limit is specifically designed to ensure decentralization and security---the former by maximizing the ability to run nodes and participate in mining, and the latter by forcing users to pay some of the cost of confirmed transactions, rather than externalizing all costs onto nodes and miners (forcing some off the network). I take exception when people try to make this about "free markets vs. central planning."
Bitcoin was centrally planned from Day 1; the entire point is trying to design the optimal system, with regard for all of its parts. The point wasn't to disregard most of the system so we can guarantee cheap/free transactions for users.
blocksize doesn't affect miners as much as nodes. Miners who can afford to mine in this market can afford to have bigger blocks too, because blocksize is limited by nodes, not miners.
And 98% of nodes right now can support 4MB blocks without a problem.