Stale has less to do with mining and more to do with propagation. If you have all the cores screaming running the SHA did you have enough CPU power left over to do the rest and get it out to the world. It's not a lot of computing power, but if you don't have any to spare it could cause issues. The other half of stale is networking. How many nodes is your node talking to? If there is a lag there and more nodes see the other block then you loose.

Also, it could be other miners deliberately causing reorgs. I don't see the point on testnet, but if I have enough power I can just ignore some blocks figuring I will get 2 or 3 while the rest of the network gets 1. That would just be a waste, but I can see people messing around to see if they can do it.

-Dave