Call For Testing: nvidia-drm kernel module

A

ashafer

Hello everyone, I'm happy to announce that the port of Nvidia's DRM driver is now ready for public consumption. Please visit the github repo to grab the latest copy of the driver. Simply follow the installation instructions and load the nvidia-drm.ko kernel module to give it a go. Please post any issues you find in the issues tab on Github.

All the details: https://badland.io/announcing_nvidia_drm.md

There are two main use cases that nvidia-drm.ko enables for Nvidia users:
  • Wayland compositors, which use DRM to perform rendering. wlroots-based compositors (i.e. sway) run pleasantly with nvidia-drm
  • PRIME multi-GPU buffer sharing in X11, where DRM is used to share buffers between an integrated GPU and the Nvidia GPU

There are two caveats you should be aware of:
  • Suspend-resume will most likely not work with wayland. See the known issues section of the blog post, but this is because the FreeBSD Nvidia driver does not have support for vidmem preservation yet.
  • Only GPUs supported by driver version 525 and later will be supported.

Thank you so much to everyone who has given feedback so far, I'm looking forward to keeping this up to date and getting it into the ports tree in the near-ish future. Please feel free to reach out with any comments or questions you might have.
 
I can report it works for me on 13.1-RELEASE when using stable/13 FreeBSD source and drm-kmod 5.10-lts source. (Nvidia 3080 laptop)
Great work!👍
 
What are details on DRM? How in this project NVidia drivers work with wlroots? I know it is expected to be GBM, but nvidia has EGL, custom implementation. I also know, that NVidia added preliminary support for wayland, but only for xwayland layer. I have strong opposition to using workarounds such as xwayland and I use only application that do work in native wayland session.
 
I have been using the driver for some time and I have good news as well as bad news:
Starting with the good news: It seems to be very stable, no crashes.
Now the bad news: I tried with Wayfire and Sway and both have flickering. As soon as e.g. chromium or other graphical tools come into the came the screen flickers from time to time (quite often actually). Googling it makes me think this is a common Nvidia and Wayland problem, anyhow I wanted to mention it, maybe there is a workaround/solution.
 
This is good news, interesting news. Unfortunately, I'd have to buy new hardware to help with the testing, so I will just follow along here :)
 
Update:
The flickering could be fixed in Sway by setenv WLR_RENDERER vulkan before starting sway.
Unfortunately Wayfire does not support Vulkan afaik, so I am trying to learn Sway atm.
 
Sorry for the late reply, but nvidia-drm works the same way on BSD as it does on Linux with respects to wayland and PRIME. For wlroots GBM will be used.

Regarding the flickering, yes this is most likely an issue with implicit sync (not BSD specific). It should only happen when sharing between intel and nvidia, so another possible workaround would be setting an optimus laptop to Discrete Graphics mode (nvidia only). Switching to vulkan is a good idea as a workaround, but last time I tested it wasn't stable.
(hacky advice to anyone messing around, but if you really want the GL wlroots to work you can put a call to glFinish() after the wlr_renderer_end call in output_render in sway)

Also please note that for now you'll need a couple changes to X, drm-kmod, and the FreeBSD source. If that's too much getting your hands dirty you can wait until they are all merged:
After this stuff gets smoothed over I'll have an update about a simple way to set up PRIME render offloading.
 
Hello to everyone.

What I would like to do is to use Stable Diffusion on FreeBSD and also to install the AUTOMATIC1111 / stable-diffusion-webui natively on FreeBSD. I'm investigating about what could be a doable method do that that,actually. I've sent an email message to ports@freebsd.org to ask more informations. A developer replied to me with the following message :


When you try to install pytorch you will get an error you should use torch name instead.. but there are no packages provided from FreeBSD..and it does not build from the sources.. more work is necessary to update the pytorch in order to build and run on FreeBSD.. looks like weeks of work.. and this should be done by someone already familiar with that tool to verify its operations even if it builds.. also note that it has some hard dependencies on hardware acceleration that may not yet be available here. I would strongly use platform that is officially supported by the project. Get the correct tool for the task.
Click to expand...

So,I can't use stable diffusion and its webui directly and natively on FreeBSD. This is the reason why I'm looking for some alternative methods to achieve that goal. One could be to install pytorch and all its dependencies as well as the stable-diffusion-webui using the Linuxulator. I don't know if it will work,anyway,if it will work,the next step would be to install your nvidia-drm driver. I know for sure that it will not work if it will be installed within the linuxulator chroot,so it should be installed natively on FreeBSD. The third step will be to install Wayland and sway natively on FreeBSD,I imagine. The 4 step could be to configure Xorg to enable the support for the Prime Render Offload and the 5 and last step would be to enable it in FreeBSD issuing more or less the following commands :

__NV_PRIME_RENDER_OFFLOAD=1 vkcube
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | grep vendor

is everything right ? What do you think about this project ? Do you think that it could work ?

I see that between the ports of my current FreeBSD installation (13.1-RELEASE-p3) I can see only the nvidia driver version 510.60.02,but on your repository there is an updated driver,version : 525.60.11. What should I do ? should I remove the 510.60.02 and install the 525.60.11 from your github ?

And I have another question to ask. Someone of you can provide a good tutorial to learn how to install Wayland and sway natively on FreeBSD ? I never did that before but I read that easily it could not work because it is a new feature,not free from bugs. I don't want to waste my time fixing bugs because I've found a tutorial that will not work.

What else ? my goal is to try to accomplish the project that I've explained and I will share the results that I will get.

These are the hardware specs of my system :

Code: 
# lspci

00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 0d)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 0d)
00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)
01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 5403 (rev 03)
 
I'm have no idea about your questions on stable diffusion, this is probably not a good thread to look for an answer for that.

Yes, you will install nvidia-drm, wlroots, sway, etc natively on FreeBSD. Please note that there are a couple open issues I am looking into regarding 13.1 support. 14.0-CURRENT should be working fine at the moment. I'm sure there are good examples elsewhere on the forums for setting up sway, although you will need the --unsupported-gpu flag. You can "lock" the currently installed nvidia-driver package, and then install the 520.60.11 driver overtop of it from github. Since these are some pretty major changes to your system I would definitely play around in a boot environment first so you can roll things back if you regret it.

Also a note, the PCI list you sent doesn't make me think that you are on an optimus machine since I don't see an integrated GPU. I see two VGA entries, for two different Nvidia cards. I don't think PRIME/Optimus/etc isn't going to do anything for you since you're only running Nvidia and this is a desktop machine. Also, although nvidia-drm should work fine out of the box on multiple GPUs, it's not something I've actually tested yet. Also if you're running wayland you have no need of Xorg as they do the same thing.
 
Mine does not want to be a stable diffusion only question. That's the goal that I want to achieve. But the procedures and the tecniques that I want and need to use for sure are tied to your nvidia-drm driver and to a lot of other features that need to be enabled to accomplish the overall project,including the testing of your driver,that I want to try to help the community. So,with that project I want to try and learn more than one topic and I would like to have your opinion on the overall project. This is the integrated GPU on the motherboard :

00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)

If I don't get wrong,this is the right profile for me.

#Use NVIDIA graphics only - gives more performance than Intel graphics, but drains more battery (which is not welcome for mobile devices). This utilizes the same underlying process as the optimus-manager and nvidia-xrun options, it should be utilized for troubleshooting and verifying general functionality, before opting for one of the more automated approaches.

or even this :
even if it makes no sense for me to use this approach. Take in consideration that I'm using the Corvin's patches,so I can passthru the Intel and / or nvidia gpus to a linux vm.

I see that the changes that I should do to my system are massive,so maybe I will make a new FreeBSD installation,this time using the 14-CURRENT or I could create a new FreeBSD 14 vm virtualized with bhyve.
 
Ah right sorry, I thought the graphics devices always showed up with the VGA prefix. It's still odd to me that you have two Nvidia cards present. Is this on a laptop?

I personally would configure it so things run on the intel GPU by default and only get run on Nvidia when __NV_PRIME_RENDER_OFFLOAD is used. Just depends on how you want to configure things.
 
ashafer said:
Ah right sorry, I thought the graphics devices always showed up with the VGA prefix. It's still odd to me that you have two Nvidia cards present. Is this on a laptop?

I personally would configure it so things run on the intel GPU by default and only get run on Nvidia when __NV_PRIME_RENDER_OFFLOAD is used. Just depends on how you want to configure things.
Click to expand...

It's a workstation. Yes,that's right. The better chose is using the Intel GPU as default and the Nvidia RTX 2080 ti for 3D graphics / rendering. Anyway there is an important difference between using a Linux VM or directly in FreeBSD and this difference is called framebuffer. For sure doing that within Linux is easier,because Linux is already ready to make work every component needed (the nvidia driver,wayland,the prime render offload,everything),but what's the best method and tool to interact with the linux vm ? I know for sure that I can't enable the vnc framebuffer together with the nvidia GPU that I have passed thru inside the vm,because I have only one good monitor. The output can't be redirected to my main monitor because it is in use by FreeBSD and It can't be redirected to the 2 monitor because mine does not work well. Which other kind of interface can I use ? I can think to VirtualGL only,but I'm not sure if the performance will be good because it is good only for the OpelGL applications and I don't know if stable diffusion can have some benefit from it. I would like to see what's happening within the Linux VM when I use the Stable Diffusion WebUI. I find it more comfortable. I don't even know if see only the web-ui is enough to work decently with stable diffusion. But It's not only a matter or personal preference,but also of how to have the better performance. If virtualGL does not support the 3d acceleration,I will get a lot of latency,that will have a bad impact on the speed that my nvidia GPU can give me when it processes the big amount of images that stable diffusion needs to elaborate.
 
What model laptop is it? I'm very curious to see an optimus machine with two discrete GPUs and a integrated GPU.
 
Trying to compile it but getting:
Code: 
In file included from /home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/linux/fb.h:4:
/home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/uapi/linux/fb.h:5:10: fatal error: 'linux/i2c.h' file not found
#include <linux/i2c.h>
         ^~~~~~~~~~~~~
1 error generated.

I would appreciate if you could point to the right direction.
 
blackhaz said:
Trying to compile it but getting:
Code: 
In file included from /home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/linux/fb.h:4:
/home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/uapi/linux/fb.h:5:10: fatal error: 'linux/i2c.h' file not found
#include <linux/i2c.h>
         ^~~~~~~~~~~~~
1 error generated.

I would appreciate if you could point to the right direction.
Click to expand...
Are you on the correct branch?
 
blackhaz said:
Trying to compile it but getting:
Code: 
In file included from /home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/linux/fb.h:4:
/home/blackhaz/git/drm-kmod/linuxkpi/gplv2/include/uapi/linux/fb.h:5:10: fatal error: 'linux/i2c.h' file not found
#include <linux/i2c.h>
         ^~~~~~~~~~~~~
1 error generated.

I would appreciate if you could point to the right direction.
Click to expand...
It might be trying to pull in the i2c.h from /usr/src, so I would make sure you have /usr/src populated and the right branch checked out for drm-kmod. Remember if you have things in non-default locations you can use the BSDSRCTOP and DRMKMODDIR env vars in the README to point the build at them.
 
You must log in or register to reply here.
Back
Top