It can be weird to think about how Bitcoin works for people just finding out about it. These hardware devices hold the keys you use to spend your Bitcoin. As long as you have these keys, you can tell the network to send your Bitcoins from your wallet to another. It is true that if you lose the device, you still have access to your coins, but this is only if you've backed up your wallet. Backing up your wallet is like making a copy of the key you use to send your coins, and putting it somewhere safe. Since this copy is usually on paper, it's not vulnerable to any sort of hack and can be safely stored in a fire safe, safe deposit box, under your pillow, etc.

You use your device to send your Bitcoins because that's where the keys are. In short, what happens is you use an online app that generates a "lock" for you (in technical terms this is an unsigned transaction). It then sends it to the device to unlock it (signing the transaction). Because your key never touches your computer, which could be infected with malware, it is much safer than a software wallet (a program on your computer instead of a hardware device).

The problem with Bitcoin is it is inherently a very technical system and can be hard to understand, but if you didn't get any of what I said just ask and I'll try to help you understand it.

An easy solution is to download "Ethereum" and create a new wallet.
It creates a "seed" which are just some random words in a specific order. You need to remember (for example store in a password manager) this seed but never share it with anyone.
Now transfer the Bitcoin funds to this new wallet.
The backup is the "seed". As long as you have that you can access the coins from any computer at any time.

Like RGBKey says it's hard to comprehend this at first. Think of it like your coins are stored in the cloud. Everyone who has your exact seed words can access them. It's secure because guessing the seed is basically impossible.