This is actually correct. Apologies for the over-simplification. Bitcoins are indeed abstractions; they exist only in the form of ‘unspent transaction outputs’, UTXOs. Allow me to try to clarify with the help of a blockchain explorer:

• One of the first bitcoin transactions was https://blockchain.info/tx/0437cd7f8525ceed2324359c2d0ba26006d92d856a9c20fa0241106ee5a597c9, wherein a 50 bitcoin UTXO was created as a reward to the miner.
• That UTXO was spent in transaction https://blockchain.info/tx/f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16: 10 bitcoins were sent to another address, and a 40 bitcoin UTXO was created as change to the sender.
• That 40 bitcoin UTXO, in turn, was spent in transaction https://blockchain.info/tx/a16f3ce4dd5deb92d98ef5cf8afeaf0775ebca408f708b2146c4fb42b41e14be. Again, 10 bitcoins were sent to another address, so this time, the amount of the change UTXO left with the sender was 30 bitcoins.

And thus it continues. Every bitcoin wallet balance consists of one or more UTXOs to which the wallet owner holds the public key. So it is actually the UTXOs that exist on the blockchain. Each UTXO is either change created from a previous transaction, or a coinbase reward credited to a miner. And even if you lose the private key of a UTXO, the output continues to exist; you just won’t be able to spend it.