Imagine a 10-band highway, and you have 1000 cars, and they all need to go from point A to point B before moving from point B to point C.
Yeah, I don't understand this. But it's ok, I don't need to. I don't understand how the speed is tied to this...the same would be true for the fastest of fastest kangaroos, using the fastest of fastest CPUs.
I know that more but slower kangaroos, will solve faster than less, but faster kangaroos, when dealing with higher bit ranges. Speed versus "efficiency / optimization" (DP overhead) are two different things in my opinion.
The highway is the on-chip registers, the parking lot is the GPU memory.
On-chip registers are insanely more times faster than reading and writing data from and into GPU memory, which is what the JLP kernel is all about. The loops are also in the bad order if what we care for are the jumps, not the kangaroos.
Maybe you wouldn't agree on this, but it is a really bad idea to dump millions of kangaroos into memory and read/jump/store them, to achieve the same efficiency as less kangaroos with a correct average jump size. Why? Because you can have a much much better efficiency by using fewer, faster kangaroos, and have the same results, but much faster.
The lower DP overhead because of fewer number of kangaroos - that is just an extra bonus, not an targeted optimization.