Nothing is ever 100% safe, even with your private keys hidden someone can physically attack you and torture you until you give them up. In some cases people feel safer with a 3rd party. Not everyone wants to be their own bank.
I agree that not everyone wants to be their own bank. And even the ones that does want, doesn't want to hold all their money by themselves.
But the idea to remove funds from exchanges is that it's much safer.