I haven't seen this problem for 2 days now. (But ofcourse I'm not watching 24/7)
All I did was unplug my second screen.
Tried playing around with the oc and the downvolt yesterday, sadly I will need to read into modding my card. I'm using https://forum.ethereum.org/discussion/10382/boysies-31mh-rom-for-rx480-red-devil-v2 right now.
To adjust the Mem Clock I need to modify the bios. I tried and my hashrate dropped from 30 to 27 and I couldnt reach 100% mem load anymore.
Increasing the voltage didnt seem to change anything, but I only watched it for a short time.
My dcri is at 30 and I get the best results there. Mem Load is nearly all the time at 100%

Thank your you help but it seems to work now anyways