All you need is a text file containing your private key and you can store it on any medium.  Some people engrave their private keys onto metal slabs and hide them as a way to have an offline and fireproof solution.  Other people write down their private keys on pieces of paper and secure them underneath door hinges or other hiding spots in their house.  It all depends on your risk tolerance and what you're protecting from.  Anywhere you can store text (flash drive, paper, metal, etc.) would all work fine.  There is no need to have any sort of wallet so long as you have your private key.  That string of characters is all you need.