From my own experience, paper wallets, or exporting the private key to a text file that is not stored anywhere online are my two favorite ways of storing bitcoin long term.  As others have already said, Exchanges come and go and you also have to rely on the good nature of others guarding those sites.  Hardware can become faulty in rare occasions.  Hopefully, you'll find a solution that's fit for your purposes.