Nvidia-xrun

From ArchWiki

Nvidia-xrun is a utility to allow Nvidia optimus enabled laptops run X server with discrete nvidia graphics on demand. This solution offers full GPU utilization, compatibility and better performance than Bumblebee.

X server can only be used with integrated graphics or discrete Nvidia graphics, but not both, so user might want to switch to separate virtual console and start another X server using different graphics from what was used for the first X server.

Installation

Install:

  • nvidia - if using nvidia-390xx - you have to edit nvidia-xrun*'s PKGBUILD and remove the nvidia depend
  • nvidia-xrun-gitAUR(recommended) or nvidia-xrunAUR (old method, uses bumblebee to switch off the dGPU)
  • Any Window manager, since running application directly like so nvidia-xrun <application> is not recommended.

Configuration

Setting the right bus id

Note: If you installed package from AUR, the bus id has been automatically set in /etc/X11/nvidia-xorg.conf. Make sure the bus ID has been correctly set, otherwise change it (you can find correct bus ID using lspci command). In some cases you need to convert the hex output from lspci to decimal for nvidia-xorg.conf, e.g. 3b:00.0 becomes "PCI:59:0:0"

Find your display device bus id: 

 $ lspci | grep -i nvidia | awk '{print $1}'

It might return something similar to 01:00.0. Then create a file (for example /etc/X11/nvidia-xorg.conf.d/30-nvidia.conf) to set the proper bus id: 

/etc/X11/nvidia-xorg.conf.d/30-nvidia.conf
Section "Device"
    Identifier "nvidia"
    Driver "nvidia"
    BusID "PCI:1:0:0"
    #  Option "DPI" "96 x 96"
EndSection

Also this way you can adjust some nvidia settings if you encounter issues: 

/etc/X11/nvidia-xorg.conf.d/30-nvidia.conf
Section "Screen"
    Identifier "nvidia"
    Device "nvidia"
    #  Option "AllowEmptyInitialConfiguration" "Yes"
    #  Option "UseDisplayDevice" "none"
EndSection

eGPU setup

You can also use this in an eGPU setup. Make sure to load the nvidia-modeset and nvidia-drm modules and add the option Option "AllowExternalGpus" "true" to the "Device" section.

Change the auto-generated configuration to use the internal display on devices with multiple NVIDIA cards: 

/etc/X11/nvidia-xorg.conf.d/30-nvidia.conf
Section "ServerLayout"
  Identifier "layout"
  Screen 0 "nvidia" 0 0
  Inactive "intel"
  Option  "AutoAddGPU" "false"
EndSection

Section "Device"
  Identifier "nvidia"
  Driver "nvidia"
  BusID "PCI:9:0:0"
  Option "AllowExternalGpus" "true"
  Option "ProbeAllGpus" "false"
  Option "HardDPMS" "false"
  Option "NoLogo" "true"
  Option "UseEDID" "false"
#    Option "UseDisplayDevice" "none"
EndSection

Section "Screen"
  Identifier "nvidia"
  Device "nvidia"
  Option "AllowEmptyInitialConfiguration" "Yes"
#  Option "UseDisplayDevice" "None"
EndSection

Remember to set the bus id to the correct graphics card.

Automatically run window manager

For convenience you can create an $XDG_CONFIG_HOME/X11/nvidia-xinitrc file with your favourite window manager. (if using nvidia-xrun < v.0.3.79 create $HOME/.nvidia-xinitrc) 

if [ $# -gt 0 ]; then
  $*
else
  openbox-session
  # Alternatively, you can also use xfce4 or bspwm:
  # xfce4-session
  # bspwm
fi

With it you do not need to specify the app and can simply execute: 

$ nvidia-xrun

Since this method starts an isolated X server, it is also a good idea to get a copy of all the other configurations files that you have located at /etc/X11/xorg.conf.d/, except for your prior standard iGPU configurations.

Use bbswitch to manage nvidia

Note: nvidia-xrun > 0.3.78 should disable the card automatically so this method is unneccessary

When the nvidia card is not needed, bbswitch can be used to turn it off. The nvidia-xrun script will automatically take care of running a window manager and waking up the nvidia card. To achieve that, you need to:

  • Load the bbswitch module on boot
# echo 'bbswitch' > /etc/modules-load.d/bbswitch.conf
  • Disable the nvidia module on boot:
# echo 'options bbswitch load_state=0 unload_state=1' > /etc/modprobe.d/bbswitch.conf

After a reboot, the nvidia card will be off. This can be seen by querying bbswitch's status: 

$ cat /proc/acpi/bbswitch

To force the card to turn on/off respectively run: 

# tee /proc/acpi/bbswitch <<<OFF
# tee /proc/acpi/bbswitch <<<ON

For more about bbswitch see Bumblebee-Project/bbswitch.

Usage

Start at boot

Enable nvidia-xrun-pm.service - this shuts down the nvidia card during boot.

Once the system boots, from the virtual console, login to your user, and run nvidia-xrun application.

If above does not work, switch to unused virtual console and try again.

As mentioned before, running applications directly with nvidia-xrun application does not work well, so it is best to create an nvidia-xinitrc file as outlined earlier, and use nvidia-xrun to launch your window manager.

Troubleshooting

NVIDIA GPU fails to switch off or is set to be default

See #Use bbswitch to manage nvidia.

If Nvidia GPU still fails to switch off, or is somehow set to be default whenever you use or not nvidia-xrun, then you might likely need to blacklist specific modules (which were previously blacklisted by Bumblebee). Create this file and restart your system: 

/usr/lib/modprobe.d/nvidia-xrun.conf
blacklist nvidia
blacklist nvidia-drm
blacklist nvidia-modeset
blacklist nvidia-uvm
blacklist nouveau

There seems to be a race condition between nvidia-xrun-pm.service and systemd-modules-load.service which loads the nvidia modules. If the latter runs first, nvidia-xrun-pm will hang (actually the tee command) during device removal. If on the other hand nvidia-xrun-pm runs first then it will succeed, and later the modules will fail to load with an error (which is harmless but ugly). For this reason it might be better to always blacklist the above modules.

DRM kernel mode setting should be enabled for PRIME synchronisation to work (for example on muxless devices where only the Intel GPU is connected to outputs). However, consider disabling it in case there is an issue. See NVIDIA#DRM kernel mode setting

On certain hardware, the Nvidia GPU exposes two devices on the PCI bus: a 3D controller and an audio device. In this case, both devices need to be removed from the bus in order for the GPU to fully power down. This can be done by simply adding a line for the audio device bus id in /etc/default/nvidia-xrun, and the corresponding line in the function turn_off_gpu in /usr/bin/nvidia-xrun to remove the second device.