Oh this is very good! I forgot to say about virtual machines.
http://www.sandboxie.com/ is very useful tool!
Yes - that's very good point! I will add it to the OP.
And also:
You could use an old computer exclusively to install wallets onto, and never let a wallet near your newest computer that you use for everything else.
Good idea!