Max GPUs i have in one rig is 13. In case this doesn't help we need to see nvidia-xconfig output and xorg.conf that it makes, so we can try to find what's wrong.
Also you can try to reflash the latest version of nvOC to usb stick, edit 1bash at windows partition (don't forget to enable headless mode) and make the first boot from usb stick without monitor attached. After the first boot in headless mode you need to wait for autoconfiguration and auto reboot. Then use network scanner or see DHCP leases on the server side to find the IP of your rig (in most cases it will be the same as in previous attempts), ssh to it and see if xorg was correctly configured.
Thank you very much for your help!
My rig is now running with all those settings: New USB, Headless, no monitor attached, know IP, have SSH and mining. Here's the weird thing: See below for my current xorg.conf that is running and I'm getting all 19 GPUs mining but only the 1st one is OC'd. What's weird about it is that:
When I run
sudo nvidia-xconfig -a --cool-bits=28 --allow-empty-initial-configuration
I get a new xorg.conf that does two things:
Bus IDs lose the "@" symbol, and mining does not work.
Any ideas why some BusIDs use the @ while most I've seen don't, and why this seems to be allowing mining to run?
Also weird is that if you notice, there are only BusIDs for 0-3 (4 and 5 are missing), then it continues from 6-11 and that's it.
What are your thoughts on this?
Current xorg.conf:
Section "ServerLayout"
Identifier "layout"
Screen 0 "nvidia"
Inactive "intel"
EndSection
Section "Device"
Identifier "intel"
Driver "modesetting"
BusID "PCI:0@0:2:0"
Option "AccelMethod" "None"
EndSection
Section "Screen"
Identifier "intel"
Device "intel"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:1@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:2@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:3@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:6@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:7@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:8@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:9@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:10@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:11@0:0:0"
Option "ConstrainCursor" "off"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration" "on"
Option "IgnoreDisplayDevices" "CRT"
EndSection
current lspci:
00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM
00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (
00:02.0 VGA compatible controller: Intel Corporation Device 1902
00:14.0 USB controller: Intel Corporation Device a2af
00:16.0 Communication controller: Intel Corporation Device a2ba
00:1b.0 PCI bridge: Intel Corporation Device a2eb (rev f0)
00:1b.5 PCI bridge: Intel Corporation Device a2ec (rev f0)
00:1b.6 PCI bridge: Intel Corporation Device a2ed (rev f0)
00:1b.7 PCI bridge: Intel Corporation Device a2ee (rev f0)
00:1c.0 PCI bridge: Intel Corporation Device a294 (rev f0)
00:1c.5 PCI bridge: Intel Corporation Device a295 (rev f0)
00:1c.6 PCI bridge: Intel Corporation Device a296 (rev f0)
00:1c.7 PCI bridge: Intel Corporation Device a297 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Device a298 (rev f0)
00:1d.1 PCI bridge: Intel Corporation Device a299 (rev f0)
00:1d.2 PCI bridge: Intel Corporation Device a29a (rev f0)
00:1d.3 PCI bridge: Intel Corporation Device a29b (rev f0)
00:1f.0 ISA bridge: Intel Corporation Device a2c8
00:1f.2 Memory controller: Intel Corporation Device a2a1
00:1f.4 SMBus: Intel Corporation Device a2a3
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connecti
01:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
02:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
03:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
04:00.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:01.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:02.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:03.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:04.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:05.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:06.0 PCI bridge: ASMedia Technology Inc. Device 1187
05:07.0 PCI bridge: ASMedia Technology Inc. Device 1187
06:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
07:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
08:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
09:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0a:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0b:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0c:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0d:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0e:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
0f:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
10:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
11:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
12:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
13:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
14:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)
15:00.0 3D controller: NVIDIA Corporation Device 1c07 (rev a1)