You could use a persistent live USB instead of a full installation on USB. But you need to permanently install the software unless you want to re-install it every time.
Whether an updated install is more secure than an un-updated install is difficult to answer.
I would just disable the network in /etc/network/interfaces by setting some dummy value. I think that wouldn't keep it from giving the annoying waiting for network message on boot?
Very nice tutorial!
I was thinking about a variation: use virtualbox to install Debian inside windows.
The virtual machine would have no network, and an encrypted lvm. I would share files between windows and Linux by sharing a host directory with the VM.
That way, I wouldn't have to reboot each time...
I realise this setup would be a little more vulnerable, but I guess it would still be reasonable.
Or am I missing something?
Every method has a tradeoff of convenience vs security. Just like in the real world you keep X$ in your wallet that you carry with you, Y$ in your safe, and Z$ in Fort Knox. Everyone has to decide for themselves what tradeoff they can live with.