@papampi re: hardware watchdog

Sorry for being brief here, but I'll try to give you some pointers.

you should look at ubuntu package watchdog and the watchdogd daemon. When implemented correctly (I hope it is) with the help of the kernel is should work as hardware watchdog timer (on mobos where the hardware watchdog can be enabled in bios or where it's enabled by default).

It works by starting watchdogd which is a daemon that will reset a watchdog (hardware bit in the cpu/mobo) once a minute. Once started, if watchdogd does not reset the timer, a hardware reset occurs.

Sometimes it's possible that there's a hardware issue with the gpus where a simple reset will not be good and you might need a physical power on-off.

hth, bytiges.