The physical keys you're referring to are the private keys (i.e. 256-bit numbers) of your addresses. Of course you can write them down (or memorize if you have a good memory). See https://en.bitcoin.it/wiki/Private_key