It is the exact same situation for any other wallet which is not Bitcoin Core or is not pointed exclusively at your own server. A popular combination with any hardware wallet is to use it via Electrum rather than it's own software (Trezor Suite, Ledger Live, etc.), but all this does is shift the point of failure from the hardware wallet manufacturer to whoever is running the Electrum server, in that the person running the Electrum server can now link all your addresses together.

The only way around this is to run your own node, and use your hardware wallet solely via your node. You can run an Electrum server on top of your node and then use your hardware wallet with Electrum, but you can also point Trezor Suite at your Electrum server directly: https://blog.trezor.io/connecting-your-wallet-to-a-full-node-edf56693b545