Quantum computers are a long ways away from becoming general purpose computers. As of right now, quantum computers are highly limited to only being able to perform specific kinds of calculations and tasks. There is no threat stemming from quantum computers at the moment and there won't be for quite some time now.
   I would worry if quantum computers became commercially available like smartphones, then there would probably be room for panic to ensue. I believe that people have been watching way too many science fiction movies.