There is a lot of work that needs to be done around the area of private key management. If you are not on top of the game in terms of it security, like multiple encrypted backups, strong passwords and multiple steps of verification, you soon see that we are far away from your average person being comfortable holding significant amounts in crypto currencies. Its not like keeping cash in a vault, but thats where we'll need to go, figuratively speaking.