Pushpool can't instantaneously notify everyone however, due to network bandwidth. Essentially, it goes through a list of all open connections and sends each the LP push consecutively. Small pools can send all their miners the notification quickly, but depending on the pool implementation and the number of open connections, this can take more than five seconds on busy pools. If you are on the start of the list, you get notification fast; at the end of the list, it can take longer. This means that the pool delay can be random per miner, much more significantly than packet transit times. Profiling your pool connection time may give different results from others, and may give different results even if you restart bithopper on a different day and make new RPC connections to the pool.
Thanks for the perfect explanation!
This is also the reason why ultimatively I want to have a stripped down Bitcoin client that watches block announcements and where they come from instead of timing longpolls in the longer run.
To get more data on Longpolls we would need to connect bitHoppers together (via IRC or something else) - something that might not appeal to some people. Also I'm not sure if it really can improve the prediction rate significantly if longpolls really differ by 5 seconds on the same pool. There are 2 interesting factors to consider: How often do we miss a block by a LP pool and how often do we think that an LP pool has solved a block when it in reality hasn't?
The first rate is not that critical, the second one can seriously harm business though.