I have my backup wallet on a Tails live USB.

 https://tails.boum.org

 First you burn a DVD. Booting from the DVD you can then use a Tails wizard to make a USB with a LUKS encrypted persistent partition.

 I can't get a miner to run on linux yet, so I do my mining on a Windows to Go external hard drive with BitLocker full disk excryption.

http://arstechnica.com/business/2012/03/ordering-windows-to-go-how-to-create-a-bootable-windows-8-usb-thumb-drive/

 I have plug and play hard drives which I only insert when I'm playing video games.