What OS are you using. If Windows, is it a BSOD?
I have a similar problem with one of my rigs. Although I didn't run memtest, I did run the built in Windows Memory Diagnostic tool (Control Panel -> Administrative Tool -> Memory Diagnostic). This test essentially froze. I swapped out the memory and now I'm hashing away. The BSOD would happen pretty quickly before.
In the past, I've also had random BSOD and crashes due to a bad HDD that even passed HDD checking tools like spinrite!
Best of luck!