The latest Windows driver (20.11.3) is NOT supported by the current version PhoenixMiner, so it probably switched to generic kernels, which are quite a bit slower.
kernel switch is probably the reason, but its not caused by driver (20.11.2 in my case) as 20.8.1 behaved the same...i installed new
one as I thought using older one is maybe causing problem.
meanwhile, RX580 4GB cards work fine with all driver versions