If you know something about Big Number calculation and the nature of GPUs, you know it's a lot more harder for the GPU to do prime test than to do hashing.
GPUs don't like branching and memory access is also limited.