Secure Elements are generally closed-source, in Ledger and elsewhere. Even Passport I think is using a closed-source SE, the same one used by other brands like Coldcard.
Coldcard secure element is close source, that means the hardware wallet is close source.
According to what I found out about Passport, it is regarded as open source, even on this forum.
Open Source
The best security is through openness, not secrecy. Passport is proudly open source – all software, electrical, and mechanical components are open and auditable. There is no hidden code or restricted information. You can even build a Passport yourself, from scratch!
Our hardware is open sourced via CERN’s Open Hardware License and our firmware via GPL. This means that Passport meets the legal definition for open source hardware. You can view the electronics here and firmware here. Mechanical designs will be posted soon.
I am not a programming, but I will like someone that knows about it like o_e_l_e_o to comment about it here. But if everything I have found out about the wallet is true, the secure element should be open source.