You could just download the wallet on a VM and then delete the whole VM when an exchange you use is added

If that's 'safe' I might give that a try ..
Could you aim me at a tutorial that explains how to do this ??
TIA
Triff ..
It's extremely safe, much safer than leaving your coins on an exchange!
I am constantly running multiple VMs (some Linux, some Windows). When a new wallet comes out, I just clone one of my VMs and download the new wallet to it in order to "sandbox" it. Basically, even if the wallet is compromised in some way, no damage can be done to your main system or other VMs because it is isolated. The trick is to use a separate VM for each wallet.
As someone already mentioned, Virtualbox is the way to go! It is totally free, you can install as many Linux distros as you want all for free, and then for Windows you can use a legit install CD if you have one or just grab an Windows ISO on the Pirate Bay

After mtgox i dont believe any exchange.