If I were you, I will be bothered about owning enough bitcoin in my portfolio if I haven't rather than thinking of what Quantum computers may do. It's a matter of if, no certainty that Quantum computers can actually do what they claim it will in the future. And even if it were to be so, the anti-quantum groups will find a solution against the attack. To every action there is always an equal and opposite reaction. That is the law of physics but it applies to real life scenarios.