Eleuthria mentioned in the IRC channel that he found the cause for the idle miner issues, which should be much better now. Apparently, the server was mistaking miner traffic for syn floods.
So, everything should theoretically be back to pretty stable now.