Optimization needed in general, so nothing can be done specially for that "high diff" job's you got. But there one cool new parameter in miner that you can try - --max-difficulty E.g. if you set --max-difficulty 1000 miner will reconnect to pool and get new job if current job received with 1G difficulty. That should help, because it reset the diff after reconnect, but still need monitoring and testing.

Also one more thing about that time without shares. As you can see in the end you have found a "high diff" share and your effort didn't change a lot(from 105% to 107% only), so you didn't lose almost anything here because you have solved huge share.