Quote from: KillrBee on March 12, 2022, 10:24:05 PM

Quote from: sp_ on March 10, 2022, 07:59:11 PM

v1.59

1. Exit if Dag validation fails many times.
2. Removed VARDIFF in the display for Non VARDIFF pools
3. New try to Prevent timeouts with debug output.

https://github.com/sp-hash/TeamBlackMiner/releases

TeamBlackMiner_1_59_cuda_11_5.7z
https://www.virustotal.com/gui/file/616ee936db716745a49eeb258229e282fe0c1e6019bedfc83e2be795ea413336?nocache=1

TeamBlackMiner_1_59_cuda_11_4.7z
https://www.virustotal.com/gui/file/f4f66a242c0d59f1058cb3505ddbf3d271681d3e281073d91681ee81172f915f?nocache=1

TeamBlackMiner_1_59_Ubuntu_18_04_Cuda_11_5.tar.xz
https://www.virustotal.com/gui/file/672e4807e307eeb881f38e0bd4bbb53047b89ff5ee4562657c72505411d8398b?nocache=1

TeamBlackMiner_1_59_Ubuntu_18_04_Cuda_11_4.tar.xz
https://www.virustotal.com/gui/file/850a9d979ec97f0523b678a7de1c026b2fd44c19c39b0e4a3aa15a9bbc95b3e4?nocache=1

Took almost 24 hours, but here's a crashed AMD GPU:

Initial Failure w/ stats:
<snip>

Parsed GPU0 output from 1st failure to crash:
<snip>

Here's another example:

00:05:52 [2022-03-14 08:05:45.276] GPU1 gfx1030 575.87 kH/W 66.23 MH/s 66.22 MH/s 156/0/0 100.00%
00:05:52 [2022-03-14 08:05:50.586] GPU1 hashtime 20c
00:05:53 [2022-03-14 08:06:43.051] GPU1 hashtime 20c
00:05:53 [2022-03-14 08:07:12.860] GPU1: 1d
00:05:53 [2022-03-14 08:07:12.860] GPU1 gfx1030 575.85 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%
00:05:54 [2022-03-14 08:07:35.516] GPU1 hashtime 20c
00:05:55 [2022-03-14 08:08:27.979] GPU1 hashtime 20c
00:05:55 [2022-03-14 08:08:47.132] GPU1 gfx1030 575.86 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%
00:05:56 [2022-03-14 08:10:05.404] GPU1: 61
00:05:56 [2022-03-14 08:10:05.404] GPU1 gfx1030 2136.27 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%
00:05:58 [2022-03-14 08:11:21.980] GPU1: ae
00:05:58 [2022-03-14 08:11:21.980] GPU1 gfx1030 2136.27 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%
00:05:59 [2022-03-14 08:12:39.996] GPU1: fc
00:05:59 [2022-03-14 08:12:39.996] GPU1 gfx1030 2136.27 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%
00:06:00 [2022-03-14 08:13:52.028] GPU1: 144
00:06:00 [2022-03-14 08:13:52.028] GPU1: has timed out, trying to restart..
00:06:00 [2022-03-14 08:13:52.028] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:01 [2022-03-14 08:15:10.236] GPU1: 192
00:06:01 [2022-03-14 08:15:10.236] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:03 [2022-03-14 08:16:26.844] GPU1: 1de
00:06:03 [2022-03-14 08:16:26.844] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:04 [2022-03-14 08:17:42.451] GPU1: 22a
00:06:04 [2022-03-14 08:17:42.451] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:05 [2022-03-14 08:19:01.088] GPU1: 279
00:06:05 [2022-03-14 08:19:01.091] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:07 [2022-03-14 08:20:19.228] GPU1: 2c7
00:06:07 [2022-03-14 08:20:19.228] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:08 [2022-03-14 08:21:33.532] GPU1: 311
00:06:08 [2022-03-14 08:21:33.532] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:09 [2022-03-14 08:22:47.305] GPU1: 35b
00:06:09 [2022-03-14 08:22:47.305] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:10 [2022-03-14 08:24:04.700] GPU1: 3a8
00:06:10 [2022-03-14 08:24:04.700] GPU1 gfx1030 0.00 H/W 0.00 H/s 0.00 H/s 157/0/0 100.00%
00:06:12 [2022-03-14 08:25:20.220] GPU1: 3f4
00:06:12 [2022-03-14 08:25:20.220] GPU1: GPU thread crashed exiting program.

The card failed at some point between 8:10 and 8:11 (you can tell by the increase in KH/W), three minutes later the card was officially pronounced dead and a restart was attempted, and 12 minutes after the restart was attempted, the card finally caused a restart of the miner. I'm not sure what the debug output is supposed to identify, but the last hashtime value was posted at 8:07/8:08.

The GPU1's stats just before the crash were stable and no values were out of anywhere near critical:

00:05:55 [2022-03-14 08:08:47.132] GPU1 gfx1030 OpenCL 0 693 27/0 80 1370 1069 115
00:05:55 [2022-03-14 08:08:47.132] GPU1 gfx1030 575.86 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%

Then, just after:

00:05:56 [2022-03-14 08:10:05.404] GPU1 gfx1030 OpenCL 0 693 20/0 80 1370 1069 114
00:05:56 [2022-03-14 08:10:05.404] GPU1 gfx1030 2136.27 kH/W 66.22 MH/s 66.22 MH/s 157/0/0 100.00%