Unfortunately, VMs are something you can use only if you don't need latest CPU technologies such as AES-NI, AVX2 etc which we heavily rely on. Also, to our experiences, VMs do not have such good uptime. There are many sporadic issues such as sudden reboots and upgrade maintenances, disks are terribly slow, private network outtages that also cause issues with storage I/O etc.

What happened should not have happened, but it did anyway. However, the chances of this happening are very very very slim, so we do not expect something like this to happen again.