Restarting the process causes some inactive minutes as the process does whatever it does as it starts up,
That's just the auto-tuning. Specify your ideal launch configuration via -l and the process will start mining as soon as it's connected. That way, and with different shortcuts right on your desktop, switching really isn't a hassle.