I think it's okay if Hardware Wallet firmware remains closed source, at some point I even agree with that approach because on another hand, even if certain company has an open-source firmware, how can you be sure that they are actually using the open-source code? Is it possible to verify in case of hardware wallets? Maybe I lack technical knowledge here.
Yes, you can and
should.
A good hardware wallet manufacturer will actually advise and instruct its customers how to download the firmware, verify its integrity and flash it. It should also make sure to have
reproducible builds; this means being able to easily check that the firmware download matches the code.
It should also be easily possible to compile it yourself, alternatively.
Honestly I find it downright malicious that Ledger's defensive message control boils down to lying about the current state of the hardware wallet ecosystem (ie. claiming that consumers always have to trust hardware wallet manufacturers while that's decidedly not the case). They are trying to normalize bad practices in terms of both security and privacy, making them the very antithesis of what one should expect from a hardware wallet company.