Very annoying to enter the API keys etc after each reboot?
To say the least. But to answer the former questions, where would you store your wallet? If you have no persistence, how would you hold the massive blockchain(s) of the coins? Why not just take the proper precautions in your browser (noscript, https everywhere, etc) and encrypt or double encrypt (im sure it's possible) your wallets, or keep your wallets in an offline pc, and have a read-only wallet on your networked pc (electrum makes this really easy).
On an exchange. Yes, I know how bad that sounds (the failure of MtGox, etc.), but I don't have another option if I want to be liquidity provider on Bitfinex. The way I figure it is that major trading sites (like btc-e) are under constant attack by hackers, but they still manage to survive without losing TOO many coins. When they do lose coins due to their own security holes, they tend to reimburse the customer. If the exchange itself survives, and my login information to that exchange isn't being monitored, then I see no reason why I should lose my money.
...that is, if the exchange doesn't pull a MtGox.
Basically, my setup would be the following:
1. LiveCD for logging into Bitfinex.
2. Google authenticator on phone for logging in.
3. Google authenticator AND phone message verification for withdrawals (where they call me and give me a code).
If the exchange doesn't fail, and the linux distro isn't bugged from the start from the source code, the computer I use to make the LiveCD, or the computers on my network (which I don't THINK is possible), then I believe this is secure. Or at least the weakest link would be the exchange surviving.