I also do something similar to this. 20 GPUs are not enough. This speed is impossible. It must be a three-digit number of graphics cards, even if you change the BIOS for the GPUs and use a  different CUDA kernel.