I am not afraid of Ledger stealing the keys.
Stealing will land them in prison, so that's not my biggest worry. But leaking the keys is a real risk.
It will only land them in prison if it can be traced back to them. Otherwise it's indiscernible whether the keys leaked or an inside job happened.
But yes, andy.arden is right that in the big picture government control is probably the largest risk.
Definitely the largest risk.
As far as anyone landing in prison, I have a feeling that in the case of Ledger specifically, internal theft would probably be easily distinguishable from some random hack--for one thing, crypto is under far more scrutiny than it was back when some of the bigger exchanges got hacked and (as far as I know) nobody suffered any consequences aside from Mark Karpeles. And the other thing that comes to mind is that if Ledger's got everyone's keys and has some kind of bookkeeping going on internally, if law enforcement or investigators for plaintiffs in a lawsuit start nosing around they'll probably be able to tell what's what. This isn't 2011 we're living in.
But oh how I do wish Ledger would go down in flames. I just hope it doesn't take down a bunch of crypto owners, however trusting they might be with 3rd parties and their private keys.