I don't know if this is related to what you're talking about, but figured I'd chime in.  For a few weeks now, my Internet connection has had horrible latency (up to 500ms) on and off when mining with two mini-rigs.  This does not occur if I stop mining.  I restart my network equipment and it's fixed for a while, but the issue comes back in short order.