Some still call this type of device a "cold wallet", although your "problem" clearly proves that this is not exactly the case. My advice would be that if it's about Bitcoin, look for a solution in one of the proven air-gapped devices, and with them you certainly won't have such a problem. And no, the price of such devices is not much higher than standard hardware wallets, although you will have to get used to doing everything via QR codes or an SD card, but that is actually an advantage, not a disadvantage.

You can read more about Hardware wallets