I've got it working now I'm using powered hubs - 20 gridseeds on the one Pi controller.
The crashes have gone away, the problem now is cgminer will stop submitting work after about ten minutes (the miners keep hashing, just no shares submitted for 30mins+). Killing it and starting a new process doesn't help. Rebooting the USB hub doesn't help. Power cycling the miners doesn't help.
Cycling the Pi seems to fix it temporarily until the next time.
I'm running with 12, so it's possible 20 is too much. Might want to also verify that the PSU is still ok by using a multimeter to check 12V rail when cgminer stops submitting shares.