Since, bitcoin doesn’t allow intermediaries, banks are worried that it will cancel out the need for their services. This kind of situation is causing serious concern and that’s because it’s affecting regulation. I guess there are both good sides and the bad sides
Your whole point is technically not quite correct
Bitcoin removes third party risks, that's true indeed, but it doesn't remove intermediaries. Someone still has to confirm your transaction, and these are miners. Of course they are not like banks. However, in a more general sense, what miners do is essentially the same what banks do. If your fiat transaction could be confirmed by any independent bank out there, the difference would be negligible