Nvidia driver breaks vncserver on CentOS 7.4, is there a work around?

Question

CentOS Linux release 7.4.1708 (Core) 
uname -r output: 3.10.0-693.2.2.el7.x86_64
NVidia driver: NVIDIA-Linux-x86_64-375.66.run

When using the Nvidia graphics card driver with the Nvidia GeForce GT 720 graphics card on CentOS 7.4 it works fine for the wired computer monitor on the console. However, when attempting to connect to the vncserver, you only get a blank black screen on connection. I have removed the Nvidia driver, and VNC works again. Apparently I just recently found out that the Nouveau driver works with VNC, but it doesn't with the wired computer monitor on the console.

Is there a work around for using the Nvidia driver and be able to get VNC to work? Possible configuration files changes or a simpler GUI to use with Gnome? Currently I'm using metacity in the ~user/.vnc/xstartup file. Or is there another good alternative to using the vncserver/tigervnc?

Answer 1

TurboVNC + VirtualGL is a good alternative.

Pros:

• Good performance.
• Works with desktop environments, that require 3D acceleration (like Gnome3), with -3dwm.
• Remote 3D rendering works even with headless GPUs.

The cons is that it might be tricky to config. I've just finished setting it up on my CentOS 7 and NVidia K80. I doubt my config is perfect, but here's a set of points I'd like to highlight:

1. Official guides (1,2,3,4) might look somewhat too long and scary at the first glance, but they are rather easy to follow. They do miss some important pieces of information, though (1,2,3,4).

2. I've used kmod-nvidia drivers from elrepo, blacklisting nouveau with two lines echo -e "blacklist nouveau\noptions nouveau modeset=0" > /etc/modprobe.d/disable-nouveau.conf, not with one single-line, like suggested in many other guides (1,2).

3. Checking the following logs is a good starting point, when something goes wrong: /var/log/messages, /var/log/Xorg.0.log, ~/.vnc/*.log. Most error messages I ran into are already discussed and rather easy to google.

4. Pay attention to the xdpyinfo -display :0 sanity check there. If it's not working, try answering n/n/n in vglserver_config and disabling selinux. Also, personally, I ended up replacing gdm with lightdm.

5. Although in general Gnome3 works OK, some weird errors do happen time-to-time (e.g. Firefox works, but opening downloaded archive with the built-in archive manager fails with an error like this). So, I ended up installing KDE Plasma, so that -3dwm is no longer needed. (But I still like TurboVNC server, cause it's fast).

Answer 2

My understanding the root cause is NVIDIA installs its own GL libraries which break other X environments.

$ ldd /usr/bin/Xvnc | egrep GL
libGL.so.1 => /lib64/libGL.so.1 (0x00007f7ed8f5b000)

A cheat is to jumper-out the NVIDIA libGL.so by directing vncserver to the /usr/lib64 BEFORE /usr/lib64/nvidia:

$ diff -cbtw /usr/bin/vncserver*
*** /usr/bin/vncserver  2018-12-08 11:07:14.871180204 -0500
--- /usr/bin/vncserver.rhel71   2014-03-10 12:17:32.000000000 -0400
***************
*** 216,223 ****

  # Now start the X VNC Server

! $cmd = "export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH ; ";
! $cmd .= $exedir."Xvnc :$displayNumber";
  $cmd .= " -desktop " . &quotedString($desktopName);
  $cmd .= " -httpd $vncJavaFiles" if ($vncJavaFiles);
  $cmd .= " -auth $xauthorityFile";
--- 216,222 ----

  # Now start the X VNC Server

! $cmd = $exedir."Xvnc :$displayNumber";
  $cmd .= " -desktop " . &quotedString($desktopName);
  $cmd .= " -httpd $vncJavaFiles" if ($vncJavaFiles);
  $cmd .= " -auth $xauthorityFile";
[merc_user@pair-1-host ~]$

This is working for me with Red Hat 7.1 and CUDA 9-2.

Answer 3

To continue working around the NVIDIA vs native graphics problem with the NVIDIA libGL issue, I have made the following cheats:

$ mv /usr/sbin/gdm /usr/sbin/gdm.bin
$ mv /usr/bin/Xorg /usr/bin/Xorg.bin
$ # make edits - show results 
$ cat /usr/sbin/gdm
#!/bin/sh
#
# workaround for libGL issue
#

LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

exec /usr/sbin/gdm.bin "$@"
$ cat /usr/bin/Xorg
#!/bin/sh
#
# workaround libGL issue
#

LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

exec /usr/bin/Xorg.bin "$@"

Answer 4

Here is my solution on a fedora 29 machine. I believe this is a distribution independent issue.
Move or copy the distribution versions of libGL to /usr/local/vnclib:

$ ls -l /usr/local/vnclib
total 596
lrwxrwxrwx 1 root root     14 Feb 14 07:11 libGL.so -> libGL.so.1.7.0
lrwxrwxrwx 1 root root     14 Feb 14 07:11 libGL.so.1 -> libGL.so.1.7.0
-rwxr-xr-x 1 root root 610208 Feb 14 07:11 libGL.so.1.7.0
$

make a /usr/local/bin/vncserver:

$ cat /usr/local/bin/vncserver
#!/bin/bash
# added because nvidia driver overwrites these
export LD_LIBRARY_PATH=/usr/local/vnclib:$LD_LIBRARY_PATH
/usr/bin/vncserver $*