I have had friends buy a few coins through coinbase and never had an issue - I did show them how to create a paper wallet and move them immediately to cold storage so that nothing is stored there.  As the saying goes, if you don't have the private keys, then you don't own the coins.  Just like at a regular bank, your money is only a bookkeeping entry until you are holding it.