Well, IMO, you do have a bit of a point about the potential "new" wealthy elite being created.
However, in all aspects of life, there are lost opportunities. If indeed BTC takes off and early adopters become very wealthy, those who did not will not be adversely affected. Just not positively affected. I am one of the people who missed the opportunity with ASICs, and such is life. That doesn't change Bitcoin's excellent utility as a medium of exchange.
One plus of the potentially upcoming BTC elite, they didn't earn their status by basically stealing from society and oppressing innovation. Quite the opposite, in fact. Should BTC take off, the early adopters (IMO, including those who risked their funds on ASIC) should be well rewarded. It was, and still is, a risky venture. They're putting their funds, time, and energy into making this network work. And if it indeed works, they should enjoy the benefits.