You could use a persistent live USB instead of a full installation on USB. But you need to permanently install the software unless you want to re-install it every time.
Whether an updated install is more secure than an un-updated install is difficult to answer.
I would just disable the network in /etc/network/interfaces by setting some dummy value. I think that wouldn't keep it from giving the annoying waiting for network message on boot?
I realise this setup would be a little more vulnerable, but I guess it would still be reasonable.
Or am I missing something?
Depends on the risk your taking, how much you've got and how much it means to you etc.
You're relying on VirtualBox here, and it's probably not ideal as a secure choice of virtualisation software, but it is true that you'll miss a whole load of dumber attacks doing it that way. You may still get hit by something half-way sophisticated though, so: risk. How much do you want.