This is not completely correct. In simpler terms, Bitcoins are stored on the blockchain and the amounts owned by the private key is assigned in the public ledger.
Your private key is what you need to access those coins from the public ledger. That is why you should never share your private key with anybody. Even if you wallet software is uninstalled, you never lose access to your coins, just install it in another device and plug in your key to regain the dashboard.
Your private keys help you accessing your bitcoin and signing transactions so that they can be confirmed by Bitcoin miners. If you don't have private keys, you can not broadcast your transactions to Bitcoin mempools before miners confirm it. If Bitcoin network is dead, your private keys mean nothing because you can not move your bitcoin to any other wallet/ address.
It's different than banks, that can be lively but you don't own any private key, and always have to wait for bank approval on your bank transfers. Leaving your money in bank account can cause entire loss anytime, for example when bankrun happens, you can not do anything if you are not too early users who withdraw money and avoid death spiral of bankrun.