It's quite worrying that it only costs $100 & brute-force speed is extremely fast. It'll be time race once attacker stole the HW wallet, assuming the attacker have similar device & the owner have direct access to seed backup.
If you don't have a hardware wallet, use an offline generated private key/seed (aka "paper wallet").
Only true if you specifically mention the usage of encrypted private key (see
BIP 38) with strong passphrase.