The fact that there is a fee to send is also a drawback in itself, as I've never had to spend any money to send money from my bank account to a friend.
Then you've never sent money to another country, and had to pay the bank's exchange rate, their conversion fee, and their transfer fee. Paying by credit card also incurs a fee (up to a few percent of the transaction), but the merchant pays it rather than the customer.
Some people also might be missing the security that banking institutions can offer if they are running a personal bitcoin wallet on their own PC and this can put them at great risk.
You call it security, I call it not having ownership of my money. I feel far less secure about my fiat sitting in a bank account and I do about my bitcoin sitting on my airgapped cold storage.
I don't remember the last time I've been unable to use a credit/debit card due to downtime.
As has been discussed above, this is a common occurrence in some countries. There have also been plenty of cases when entire processors, such as Visa or Mastercard, have gone down, either across entire countries or even continents.