i don't understand why you don't trust hardware wallet. There are some hardware wallet which is fully open source, where you can even build one by yourself.
I can think of a few reasons. The most important one: it's a black box. I have to trust the manufacturer to do what they say they do, and
sometimes they turn 180 degrees.
Fair point. Although IIRC Ledger was never fully open source, partially due to closed-source secure element.
There are some hardware wallet which is fully open source, where you can even build one by yourself.
Who's really able to do that?
Someone with some electronics and computer skill. I've seen few people build their own Trezor and Blockstream Jade.