That would be my guess. And my next guess is the decryption keys are only known to Ledger and/or their "storage partners". That makes me wonder who has a backup, and how they were created. One way or another, it adds a risk that shouldn't exist in hardware wallets.