Instead of booting a live cd, you could also mount a ramdisk (or make a folder in /dev/shm) and create a symlink from it to ~/.bitcoin, then create your wallet, addresses, backup etc.. This, of course, assuming you're already running some Linux distribution. Then the files will only have ever resided in RAM; unmount the ramdisk (or delete the folder you created in /dev/shm), and the symlink to it, and it'll be almost as if the files were never there.

Just don't forget to make the backup, since otherwise--because it is in RAM--once you shut down it'll be gone forever.