Honestly I find it downright malicious that Ledger's defensive message control boils down to lying about the current state of the hardware wallet ecosystem (ie. claiming that consumers always have to trust hardware wallet manufacturers while that's decidedly not the case).
Putting aside the open-source vs closed-source war, I think the trust lies in the fact that the developers and security experts did their job properly to not mess up the code or introduce vulnerabilities that someone can exploit. That's what most people have to trust because most of us don't know how safe a code is whether we can view it publicly or not.
Trezor's open-source code means very little to me because I can't go through it and I don't understand what it does. I still have to trust Trezor and everyone that has verified the code that it's bulletproof and can't be abused. That's the trust part.
Yes and no. Ledger is deliberately setting up a false equivalence of trust.
Yes, there's always a certain degree of trust required: If you can verify the code, you still need to trust the compiler. If you can verify the compiler, you still need to trust your CPU. If you can verify the CPU, you still need to trust the laws of physics.
But.
Contrary to what Ledger is trying to sell, trusting a single company to do the right thing is not even remotely the same as having thousands of developers and hackers -- independent and contracted alike -- making sure that there's nothing fishy going on. It simply isn't.