If someone wants to help, we need test cases for the cuda mpz functions to be sure that the cuda mpz library works (it's not mine). Also debugging this with gdb (the one from nvidia that can step through gpu execution) would be insightful, to see at which code line it crashes.
Sadly I haven't the faintest how to work with GDB. Also, adding on to my root access offer, I just remembered that Amazon offers GPU instances! For the life of me I cannot find the GPU option in the instance list though. If anyone gets back to me about this, I can happily offer you a donation to pay for a bit of that instance or I can try to set it up for you.
https://aws.amazon.com/ec2/instance-types/CG1 Instances are backed by 2 x Intel Xeon X5570, quad-core with hyperthread plus 2 NVIDIA Tesla M2050 GPUs