A big fan of VM's myself, how about getting a VM (Used to use VMWare, but VirtualBox fan these days) of your favourite distro up and running, install all of the necessary software, take a "snapshot" of it (shut down is safer, but you can "snapshot" a running VM). Do your wallet transactions inside the VM after booting it up, transfer the file to cd/usb stick/whatever..., shut down the VM reverting to snapshot. Then you won't be needing to install the software every time, but you still get the benefit of having a system which doesn't actually contain your wallet. If you are extremely paranoid, download the software installs separately and install them in the VM while not connected to the web.
I'm not sure what the VM gains you.
Like the live cd, your "wallet" is NOT on the disk (in this case image). Any traces of it are disposed of when reverting to a snapshot. Unlike the live cd, you only have to install the bitcoin software once. If you really know your stuff, install the software and create your own live-cd with it.