The only different between Normal USB (holds the key) and Hardware wallets, that hardware wallets have extras such as Connect to Wallet by Wi-Fi or Bluetooth, and Built-in software which communicates with your wallet.
Hardware wallets are very different from your USB. Hardware wallets contains the necessary components which is used to generate the signature using the private keys stored on it. LiveUSBs cannot do so and it relies on the computer for the signature which makes it vulnerable to BIOS root kits as well though that would be more complicated.
If you're talking about something like an OpenDime, it generates the key on the device itself and gives you the address. You will reveal the private key only if you poke the seal.