Did you consider the learning curve and time it will take to train people to program a quantum computer, the time it will take them to code a bitcoin brute-forcer and the time it will take to test it? That gives us a few extra years of leeway, counting from the date that a quantum computer is commissioned for public commercial general-purpose use.
We don't have to wait for QC to be available for the general public to become a threat to Bitcoin's security. It should be considered critical as soon as a single government or corporation has access to a powerful-enough quantum computer.
Remember, Bitcoin's security proposition is that no single entity has the power to compromise it. That goes out the window as soon as the technology becomes available, even with limited access. But luckily even that is still many years out.