I checked all processes and everything is running fine, so it is just variance. It happens sometimes and is very noticeable on a low hash-rate pool unfortunately.