Hookup an oscilloscope to the vccint, close as possible to the fpga.
Y'know, I was never any good with an oscilliscope. One of these days
.
-Make sure new midstate load etc doesn't results in spikes.
Check. I deliberately don't stop the rings when loading nonces for this very reason; I just let garbage fly out the back end due to half-loaded work. The noise caused by that huge change in power consumption is not worth it.
-Stagger the rings start time/midstate load/nonce wrap
-Use phase offset to interleave clock transitions for the different rings
Well, they're on different clocks. However, I will build one where they all use the same clock so I can try this -- good ideas.
-Ramp the clocks up gradually from idle
Yes, already doing this.
It could also be the PLL suffering from too much noise. Try changing the loopfilter/bandwidth of the PLL.
Since it's only a jitter filter I have it on the lowest bandwidth setting.
I'm also going to try dropping it altogether after finding a comment by Austin saying that Xilinx's PLLs are very sensitive to activity in nearby logic
Might be hard but try an external high-speed clock source (connection/termination to the board is critical)
Unfortunately I don't have boards that can do that (SMA connectors, right?)