check every power connection (sockets on riser, psu, card)..minor burns makes the gpu power draw smaller. degraded thermal pads and thermal paste makes gpu components to throttle down to protect the components from overheating thus lower performance.
Well the 3070 is brand-new and I'm not using risers. I've checked temps with HWinFO: main GPU is at 65°C, hot spot is at 76°C... so I don't think it's a thermal issue. 
Graphics cards have different type of memory, some have Hynix, some have Samsung and some have Micron, on my graphic cards I get a better hashrate when I turned down my Hynix Core clock to 0, this is just normal downclocking and it has no bad effect on the GPU.
Hold on are you talking about the GPU (core) clock or the memory clock ? I am a bit confused.I've seen many users sharing their afterburner settings with very low core clock (-500MHz) and very high memory clock (+1000MHz), getting about 60mh/s. Yet the issue is that my hashrate drops significantly when decreasing the core clock.