Thank you for all of you guys help. I did some more testing and have gotten different results. I have done this as miner29 suggested,
-Made sure you have all the molex plugged into motherboard. 2 of them.
-Made sure the first gpu is in the long slot, the 16pin one
-I am using an Intel Celeron G3930 and have 8GB of RAM
I am also running HiveOS to mine for Ethereum.
I got the 3 RX5700 working. But for some reason, I cannot get any of the RX580 to work. I do not think its the risers because I swap them around and I still only get the RX5700 to work. So if I have the 3 RX5700 connected, and I connect one RX580. It does not show up on HiveOS, only the RX5700 do. Which if find strange. I can get the RX5700 working, but never 2 of the RX580. I can only get one RX580 to work if I unplug the rest of them.
I also did what stoniestfool suggested
-Switched all the PCI to gen 1
-Disabled virtualization
-Enabled 4GB
-VT-D Disabled
- 1st gpu should be in pcie 3.0 16 slot. Then the second goes to the black slot closest to cpu. Then fill up the black slots before using any of the white ones.
I have 2 power supplies. One 600W one for the MB and 2 580s and a 1200W server PSU for the rest of the cards. And I dont think its the mobo because its band new.
What I am trying to figure out now is why none of the RX580 work together.
Thanks