It's even worse than that. The thief would get your coins AND your personal info, because Ledger's system connects your KYC directly to your coins. So, if the thief has reason to suspect that you have even more coins hidden behind a passphrase, he knows who you are and where to find you, all thanks to Ledger.
That is a possibility. However, the data leaks we saw from Ledger and other companies never included any addresses or xpubs. The amount of coins never leaked anywhere which means those who still have those databases can't know who owns what. But since their names are on a list of hardware wallet users, it's reasonable to assume they have coins whose keys they believe are worth protecting with such devices. In my view the best setup for your stash is a multisig wallet with at least two airgapped co-signers, say Passport and ColdCard MK4 (or coming ColdCard Q1).
Wasn't there talk about hardware wallets not always being the best choice in multisig systems? I think I have heard both Ledger and Trezor being mentioned as problematic. Or perhaps I am thinking of a specific multisig use case and not all of them 