So, how do we verify that the hardware doesn't generate predictable entropy, regardless of whether the wallet developers have bad intentions or not?
Unless you have capability to understand the design of the hardware or at least perform randomness test[1], you either trust the hardware company or someone who audit the hardware. I'm fairly sure there are decent amount of cryptography auditor, but i've no idea how common auditor for hardware design/architecture.
[1]
https://en.wikipedia.org/wiki/Randomness_test#Specific_tests_for_randomness