Has anyone found out what exactly is triggering the driver issue that causes the hashrate drop?

I've tried everything I can to get the hashrate to stay up that was suggested.  I am running cast XMR and have 4 cards on my system: GTX 1070, RX570, and 2 Vega 56's flashed to 64. 

The nvidia card is hooked up the monitor.  I get 1800/1950 on both of my cards when I start it up just fine, but like many others lose the speed shortly after. 

I've got my power settings set to never turn off (even the monitor is not turning off at all), I've got everything that resembles crossfire or ULPS in registry disabled, my pagefile is set to 60gb. 

The only thing I don't have is a fake dongle, but the AMD cards aren't even hooked up to the monitor if that matters.

I've got my fans at 3000rpm, but my temps always seem fine.  I am usually under 60c on the core temp.  I haven't checked recently, but I believe that puts the HBM2 ~10C higher. 

I really wish I knew exactly what I was fighting, but I just seem to be trying a bunch of random things to fight this issue without any real idea of what I'm trying to correct.