Public bug reported:

DGX Station with NVIDIA V100 is installing by default the open variant
of the nvidia drivers that is not compatible with this GPUs.

This leaves the machine unable to use the capabilities of the GPU
entirely. This was previously seen in Resolute SRU testing but it now
seems to impact other releases such as Jammy and Noble


Distro: Ubuntu 22.04 (jammy)
Kernel: 5.15.0-191-generic
ubuntu-drivers-common: 1:0.9.6.2~0.22.04.11
nvidia driver installed: nvidia-driver-595-server-open


Steps to reproduce:
```
sudo ubuntu-drivers devices
sudo ubuntu-drivers install
sudo reboot
```

Workaround:
Do not let ubuntu-drivers decide automatically the recommended driver, instead 
install with:
sudo ubuntu-drivers install nvidia:580


Ubuntu Drivers output:
```
sudo ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:02.0/0000:05:00.0/0000:06:10.0/0000:08:00.0 
==
modalias : pci:v000010DEd00001DB2sv000010DEsd00001219bc03sc00i00
vendor   : NVIDIA Corporation
model    : GV100GL [Tesla V100 DGXS 16GB]
driver   : nvidia-driver-565-server-open - distro non-free
driver   : nvidia-driver-580-server - distro non-free
driver   : nvidia-driver-595-server-open - distro non-free recommended <= open 
variant should not be recommended
driver   : nvidia-driver-595-server - distro non-free
driver   : nvidia-driver-580-server-open - distro non-free
driver   : nvidia-driver-565-server - distro non-free
driver   : xserver-xorg-video-nouveau - distro free builtin
```


The following service is in failed status:
```
Found 1 failed units

Failed units:
  UNIT                        LOAD   ACTIVE SUB    DESCRIPTION
● nvidia-persistenced.service loaded failed failed NVIDIA Persistence Daemon

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
1 loaded units listed.

Logs for nvidia-persistenced.service:
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: Now 
running with user ID 129 and group ID 136
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14543]: 
nvidia-persistenced failed to initialize. Check syslog for more details.
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: Started 
(14545)
Aug 17 01:15:15 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Control process exited, code=exited, 
status=1/FAILURE
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: Failed to 
query NVIDIA devices. Please ensure that the NVIDIA device files (/dev/nvidia*) 
exist, and that user 129 has read and write permissions for those files.
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: PID file 
unlocked.
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: PID file 
closed.
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: The 
daemon no longer has permission to remove its runtime data directory 
/var/run/nvidia-persistenced
Aug 17 01:15:15 nvidia-dgx-station-c25989 nvidia-persistenced[14545]: Shutdown 
(14545)
Aug 17 01:15:15 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Failed with result 'exit-code'.
Aug 17 01:15:15 nvidia-dgx-station-c25989 systemd[1]: Failed to start NVIDIA 
Persistence Daemon.
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: Starting NVIDIA 
Persistence Daemon...
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: Verbose 
syslog connection opened
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: Now 
running with user ID 129 and group ID 136
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14570]: 
nvidia-persistenced failed to initialize. Check syslog for more details.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: Started 
(14571)
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Control process exited, code=exited, 
status=1/FAILURE
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: Failed to 
query NVIDIA devices. Please ensure that the NVIDIA device files (/dev/nvidia*) 
exist, and that user 129 has read and write permissions for those files.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: PID file 
unlocked.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: PID file 
closed.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: The 
daemon no longer has permission to remove its runtime data directory 
/var/run/nvidia-persistenced
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14571]: Shutdown 
(14571)
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Failed with result 'exit-code'.
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: Failed to start NVIDIA 
Persistence Daemon.
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: Starting NVIDIA 
Persistence Daemon...
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: Verbose 
syslog connection opened
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: Now 
running with user ID 129 and group ID 136
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14611]: 
nvidia-persistenced failed to initialize. Check syslog for more details.
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Control process exited, code=exited, 
status=1/FAILURE
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: Started 
(14613)
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: Failed to 
query NVIDIA devices. Please ensure that the NVIDIA device files (/dev/nvidia*) 
exist, and that user 129 has read and write permissions for those files.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: PID file 
unlocked.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: PID file 
closed.
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: The 
daemon no longer has permission to remove its runtime data directory 
/var/run/nvidia-persistenced
Aug 17 01:15:16 nvidia-dgx-station-c25989 nvidia-persistenced[14613]: Shutdown 
(14613)
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Failed with result 'exit-code'.
Aug 17 01:15:16 nvidia-dgx-station-c25989 systemd[1]: Failed to start NVIDIA 
Persistence Daemon.
Aug 17 01:15:17 nvidia-dgx-station-c25989 systemd[1]: Starting NVIDIA 
Persistence Daemon...
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: Verbose 
syslog connection opened
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: Now 
running with user ID 129 and group ID 136
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: Started 
(14653)
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14651]: 
nvidia-persistenced failed to initialize. Check syslog for more details.
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: Failed to 
query NVIDIA devices. Please ensure that the NVIDIA device files (/dev/nvidia*) 
exist, and that user 129 has read and write permissions for those files.
Aug 17 01:15:17 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Control process exited, code=exited, 
status=1/FAILURE
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: PID file 
unlocked.
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: PID file 
closed.
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: The 
daemon no longer has permission to remove its runtime data directory 
/var/run/nvidia-persistenced
Aug 17 01:15:17 nvidia-dgx-station-c25989 nvidia-persistenced[14653]: Shutdown 
(14653)
Aug 17 01:15:17 nvidia-dgx-station-c25989 systemd[1]: 
nvidia-persistenced.service: Failed with result 'exit-code'.
Aug 17 01:15:17 nvidia-dgx-station-c25989 systemd[1]: Failed to start NVIDIA 
Persistence Daemon.
```

** Affects: ubuntu-drivers-common (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2163716

Title:
  ubuntu-drivers install incompatible driver variant for NVIDIA V100

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-drivers-common/+bug/2163716/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to