Yes, sometimes the work gets very low (I usually get more work when it gets down to the 50s though), and since I have 11 devices all on this one worker, I sometimes don't get enough new work queued up fast enough.
I guess it drops to 0 for a short moment when there is a block change (long poll). Maybe I made the code for the Icarus a bit too sensitive to that and it complains quickly.
I'll add Stratum support soon, which will eliminate that moment of rushing to get new work after a block change.