Mining will never touch your latency. I tested 5 antminer s9 doing solo and it still didn't touch my 6mbps at&t uverse. Something else is going on with either router or modem.