Thanks for the replies guys. I added the following and it seems to be catching up pretty quick now

-s 1 --expiry 25 --queue 0

I suppose the expiry needs to be little less than average block time (which is 30 seconds for infinitecoin and 2.5min for litecoin)