I think cash transactions will also have anonymity. When you trade paper money with the other party, the other party does not know any information about you. But if you pay electronically, the other party may know your information.
However, whether it is Bitcoin or cash, it needs to be properly stored. If you lose your bank card in cash, you can retrieve your bank card through your ID card. But if you lose your Bitcoin key, you can't get your money back.
That's the dilemma of some crypto users, the thought of losing or misplacing their wallet keys.
On the other hand, a lot are still trusting the bank, because if you lost the ATM card or your financial book, you can always report it to the bank.
And then, you only need your verification IDs to replace the card or the book. This is why even some people here can't get rid of their bank accounts.
Also, some of financial transactions still do require bank details, and so most of us continue to use the banking services.
In terms of security, I think, both have their pros and cons. It is up to you how you will handle your accounts for security purposes.