I guess with a bank, it's the same as not having your Private key. If you don't have your cash or your private key with you, you're not in control of your money, someone else is.
Exactly this. Society nearly forces you to have some sort of bank account (receiving wages in cash in even disallowed in my country) but try to not give them too myuch of your wealth as they can do with it as they please.