But the other element of the story is privacy vs security. A security leak is more dangerous than a privacy leak. I mean, leaking names, emails, and addresses is bad, but leaking seeds or private keys and having your device manipulated equals the loss of coins.
It's even worse than that. The thief would get your coins AND your personal info, because Ledger's system connects your KYC directly to your coins. So, if the thief has reason to suspect that you have even more coins hidden behind a passphrase, he knows who you are and where to find you, all thanks to Ledger.