GPUs are of no concern.  The system was ingeniously designed to adapt.  What worries me is that if mining becomes unprofitable, miners will leave and the network will lose it's security.  Heck, I have my biggest mining rig offline right now, and I am not the least worried.