I have a similar issue but with Scrypt on multipools (Clevermining, in this case). Nwools noted that it's a known issue with BFGminer that the hashrate will drop as the multipool switches blockchains. I'm using a mix of Gridseed Orbs and a Blizzard. I'm not sure if this has been fixed in version 5.0.
Are you using a multipool, Dimoza? If yes, that might be issue.
actually, not. I use classical stratum "single" pool.
But... concerning to your issue, does your hashrate drops to 0 or just getting relatively lower? Cause mine drops to zero.