I have about 36% CPU utilization and 46% memory utilization out of 8GB. I also noticed while testing individual groups of GPU's using the -di parameter to find the best -dcri values for blake2s, ETHman wasn't dropping the connection. As soon as ~10 GPU's are running the connection starts dropping. Could it be a problem with indexing?
You can start two miner instances with -di 012345 and -di 6789ab to confirm this idea.
As soon as OpenCL works with 12 GPUs, system starts to work slowly with sockets accepting.