this is confusing. I'd think that 'ms' is the time measured from the moment you submitted the share till you got response from the pool. and if so it wouldn't probably have to do anything with the miner version, but the network conditions. also obviously when the block time is short your ping better be fast.
Let's try another explanation.
If your computer is running basic addition........
... meaning the lowest difficulty setting on that port is a lot higher than 3334.
This is all fascinating, seems that you're on the right track - a bit more learning and you'll understand how it all works.
However my point was that the EdjOne's claim is likely an absurd one.
And the same goes to you on your difficulty talk. EDIT: Meaning - it has nothing to do with the poll response time.