As the ASIC miners start mining, and "outdated" miners go offline, does this not pose a risk to the network in the sense that there are fewer overall miners?

The network as it stands now is far more distributed amongst a greater number of computers & hardware mining. If the majority of these go offline due to obsolescence, in essence there would be a centralization of power to a far fewer number of miners.

Although the network hash rate will be significantly higher, the number of people involved will greatly reduce. Does anyone see a risk in this? Perhaps if the ASICs develop hardware issues in the future (I hope not) and cease mining operations, I would think the network would be very vulnerable to an attack. Granted, difficulty would drop and "regular" miners would return in a balancing act.

Is the efficiency and hash rate increase worth the trade off of a less distributed mining network?