Interesting. I have a Ryzen 1700 as well with a persistent OC of 4GHz that does up to 600 h/s with the current XMR algo with no issues whatsoever but only getting 670 h/s on randomwow. From what I've gathered, this should be doing around 4 kh/s.
Someone else was having issues with overclock. Have you tried stock speeds?