In the long term I think quantum computers will crack the elliptic curve cryptography that bitcoin's based on. If good enough quantum computers are invented then bitcoin's basic code will need changing round to something quantum computers can't crack. That change of code base will render all mining hardware obsolete, and a new generation of hardware will need inventing to replace it.
Such a situation could permanently break both bitcoin and its miners if new mining hardware is not invented quickly enough.
If/when quantum computers become a threat to Bitcoin, the same threat will apply across the financial sector. I doubt that that threat would stand for long: even if Bitcoin devs don't invent a solution, others will and it'll soon be implemented in Bitcoin.
Edit:
the FAQ on the wiki discusses QC, its risks, and how they're mitigated.