After killing myself for the last 24 hours troubleshooting 2 r815's I picked up, I have learned a thing or two.
If the cover is not on the box, it will not produce full power. Board will limit CPU power evident by power drawn at plug and performance in hash. I chased my tail for quite a bit yesterday and today until I figured it out.
Thanks for this tip.
I got my feet wet and installed Ubuntu on a Dell R815 (4x Opteron 6234's). Then did all the updates, configurations and XMR-Stak Compile and it produces about 1560 H/s on the four processors and another 240 H/s on the single GTX 750 for a total of 1800 H/s.
On the second R815 with the exact same configuration I was only getting an initial 1200 H/s on the processors which dropped to 1050 H/s in a few hours. On this system the front display was orange and it was saying "system is powered but cover is off" or some such words. That reminded me of your post on the hash rate drop with the cover off.
The cover was on the system so I looked at the intrusion switch on Riser 2 and it turns out that the post on the switch was broken off so it was always telling the system that the cover was off even though it wasn't. I fixed it with a tie-down keeping the switch always pressed.
Front display is now blue and processor hashes are now steady at about 1560 H/s.