You know it wrong. A proper hardware wallet allows you reset to a new seed as many times as you want.

However, your point is somewhat correct. A hardware wallet can have loophole and the seed be created in a way they can easily find it.
Most go by trusting the manufacturer, since they may not want to risk their business and a massive lawsuit if they do fishy things.

However, the best way to fight that is use open source hardware wallet. Then you can expect that many smart people will review it.
And yet some hardware wallets are closed source...



My opinion is that if you get to have such amount of bitcoin you fear the hardware wallet company may steal them from you, the best call is to make your own cold storage.