If there is a possibility that all (or almost all) transactions could potentially have their privacy reversed then it will be as you said most concerning from a user perspective.
Even if that's true, it still depends on who you're hiding from. If you don't want your local barista to know you're rich in Bitcoin, you're still good. There's still no blockchain evidence linking your addresses.
the authorities will be focused on big criminals
I haven't seen anyone who's against that.