I might have been mistaken: the git HEAD version uses satoshis for this parameter (which means it probably is faster than 0.8.1 even with defaults parameters). I converted it to satoshis and my latency is back to <0.10s.