Depends who "people" are. If only malicious miners have quantum rigs, then yes. But incentives work in the direction that eventually every miner will have quantum rigs in this case, it simply becomes a new hardware generation.

In addition, there may be a noticeable difficulty jump in this case, but the first Grover-capable quantum miners will have relatively slow machines, just like with conventional hardware. And thus they wouldn't be able to dominate mining forever.

A recent paper by a quantum computing firm has "found out" (take that with a grain of salt;)) that hardware costs would be even more relevant in the case of quantum mining, and electricity consumption in contrast could become less important. That would hint that mining could become even more centralized on the big mining companies, and home/small-scale mining would more or less die -- until quantum computers become cheaper, then the battleground becomes leveled again. If there's no technology even faster than quantum computing ...

There's also a "last resort" option: one could request a Proof of Stake during some time (via hard fork, like in the case of Ethereum's "Merge") if really the situation becomes very unstable, e.g. if North Korea was the only actor with Grover-capable quantum computers and the rest of the world (including private companies) wouldn't have access to this tech. That's however a quite unlikely scenario.