Also, the P2Pool block time is 10 seconds, so having a latency of >10 seconds means that when a new block is detected, you're going to completely miss the next block, which would half your actual effective rate.
Finally something that is factual. Thanks!

The highest I saw mine was 3s. And as I stated multiple times, restarting bitcoind put it back down where it was supposed to be, but it did start climbing back up. I still fail to see how transactions have anything to do with it when restarting bitcoind lowers the latency.
M