Then, public keys are known by all wallets, and if any of them is compromised, then it will show you a different address. Each public key should be known by each wallet, the only difference is which private keys they have.
What if you can't store the public keys of all cosigners on the hardware wallet?
Or can this be stored at every HWW?
Would you then see the wrong address on all devices?