What you might want to do is enable onboard graphics, reason why your miner intensity is dropping cause you are remoting on that miner/video card and slowing it down to show the remote desktop, especially if you are using windows 10 due to hardware acceleration.

So if you use the onboard video card as the primary you'll be fine, that's what i am doing

Icon