Banks set up to store fiat. Exchanges are created for buying and selling crypto coins. Hardware is created and designed to hold the coins.
All are in their respective functions of the 3 objects mentioned in this topic.

As for who to trust? I leave it up to each user because so far I have placed each of the three (banks, exchanges and hardware) in their respective functions.
And yes. Don't store coins on exchanges. Delete all important data on devices that can be accessed by them such as email and so on.