bytemaster still hasn't responded further to my logic that the GPU can mine much faster (specifically he hasn't stated why he believes a hash table can't also be implemented on the GPU). He said he is sick.
GPU's will be able to generate the SHA512 hashes quicker, no doubt. But the GPU resistance relies on the GPU not being able to store them. So if your GPU is generating 10GB/s but can only store 1GB/s, there's no advantage to an i7 that generates 1GB/s and stores 1GB/s. Whether that's through the medium of a hashmap or not is not important.
ByteMaster really wants to know if there is a way to beat this with GPU's - that's why he's put up the bounty.
GPUs aren't limited to 1GB of memory, many have closer to 6 - 10GB now.
The salient issue (unless I've misunderstood the algorithm) is that he asserts a CPU can use a hash table but a GPU can't, thus he asserts the GPU needs N ^ 2 memory. I don't see why the GPU can't use the same amount of memory as the CPU? The GPU can compute 1000s of lookups on that memory in parallel, thus obviating the random memory latency to be bound at the superior memory bandwidth of the GPU.