I noticed the issue again with the miner seeming to drop hashing speeds on cards by around 100 H/s per Vega 64 card. I looked back through the mining log and didn't see anything strange. I tried resetting the cards while the miner ran (Overdriventool -r and then -p) and that didn't fix the issue. I closed the miner down and restarted it. That caused all of the cards to go back up to where they were at originally.
setx GPU_MAX_HEAP_SIZE 100
setx GPU_MAX_USE_SYNC_OBJECTS 1
setx GPU_MAX_ALLOC_PERCENT 100
setx GPU_MAX_SINGLE_ALLOC_PERCENT 100
OverdriveNTool.exe -consoleonly -r1 -r2 -r3 -r4 -r5 -r6 -p1VEGA64 -p2VEGA64 -p3VEGA64 -p4VEGA64 -p5VEGA64 -p6VEGA64
timeout /t 5
SRBMiner-CN.exe --config monero-config.txt --pools monero-sxmr-pools.txt
pause
[Profile_0]
Name=VEGA64
GPU_P0=852;900;0
GPU_P1=991;900;0
GPU_P2=1084;900;0
GPU_P3=1138;900;0
GPU_P4=1150;900;0
GPU_P5=1202;900;0
GPU_P6=1212;905;0
GPU_P7=1408;925
Mem_P0=167;900;0
Mem_P1=500;900;0
Mem_P2=800;900;0
Mem_P3=1095;905
Fan_Min=3000
Fan_Max=4900
Fan_Target=75
Fan_Acoustic=2400
Power_Temp=85
Power_Target=0
[General]
MainWindowLeft=538
MainWindowTop=277
I just lowered it to 1095 from 1100 on the memory.