How to install the nvidia-drm-kmod for the driver nvidia 525.78.01 on FreeBSD 14.2 to be able to run ComfyUI.

ZioMario · May 4, 2025

Hello to everyone.

I'm trying to run ComfyUI on FreeBSD 14.2 natively,using the miniconda installer and the appropriate nvidia driver + nvidia-drm-kmod. I would like to downgrade the nvidia driver shipped by default by the 14.2 (vers. 570.124.04) because I suspect that it does not work with the libc6-shim. Probably I should reinstall FreeBSD 13.1,where I'm sure that everything may work good,but it is very old and it went EOL,so the idea here is to use the 14.2 but with some ports that come from the past (like the nvidia 525.78.01 driver,the drm-kmod associated and maybe an old version of the libc6-shim). So. I have resumed an old post :

https://www.reddit.com/r/freebsd/comments/10ysy6m/how_to_install_on_freebsd_131release_a_different/

and I have installed the nvidia driver 525.78.01 succesfully. But now I should figure out how to install the nvidia-drm-kmod package or port associated with it,because the official one shipped with FreeBSD 14.2 (nvidia-drm-kmod-570.124.04) does not work at all. The error that it gives when I try to kld load it is :

Code:

KLD nvidia-drm.ko: depends on nvidia_modeset - not available or version mismatch
linker_load_file: /boot/modules/nvidia-drm.ko - unsupported file type

So,where I can get the missing piece of software that I need ? thanks.

ZioMario · May 4, 2025

With my big surprirse,the old FreeBSD 13.1 offers the new nvidia-drm-510-kmod-570 module ? How can this be possible ?
I need the nvidia-drm-510-kmod-525 or something like this.

W.hâ/t · May 4, 2025

It's in 14.2 too, just broken at the moment - https://portsfallout.com/fallout?port=graphics/nvidia-drm-515-kmod$

ZioMario · May 4, 2025

it says : 570.124.04.1402000_1 : it is too new,I don't need this,but a version which works with the nvidia driver 525.78.01 ; some years ago the 570.124.04.1402000_1 didn't exist at all.

T-Aoki · May 4, 2025

ZioMario said:
it says : 570.124.04.1402000_1 : it is too new,I don't need this,but a version which works with the nvidia driver 525.78.01 ; some years ago the 570.124.04.1402000_1 didn't exist at all.

You should already know how to override versions forward and backward, as you posted the very next one.
And another.
There should be more even limiting posts by me.

Note that this doesn't work for graphics/nvidia-drm-*-kmod prior to 550 series of driver, as current form of them relies on ashafer's codes included in upstream tarball and this was started from 550 series.

Prior to this, 535 series was the first one that graphics/nvidia-drm-*-kmod supported. From this through before 550 series, ashafer provided required distfile by himself. Anyway, 525 is not supported at least in-tree.

If you want to get back to 525 series, resign from using graphics/nvidia-drm-*-kmod and switch back to x11/nvidia-driver alone (or with x11/linux-nvidia-libs).

The list of 525 series I've tested to submit patch when they were the latest version is as follows. I'm trying to at least build on latest stable branch at the moment (now stable/14) when I find that nvidia released new production, new feature or beta drivers to confirm simple version overriding works or not, and if not, start working on providing patch to make them work, with 100% volunteer basis (no sponsors at all). So I've at least built and packaged below once per version.

525.78.01
525.85.05
525.89.02
525.105.17
525.116.03
525.116.04

Picked from commented out NVIDIA_OVERRIDE_VERSION= lines in my /etc/make.conf.

ZioMario · May 4, 2025

You should already know how to override versions forward and backward, as you posted the very next one. And another.

Sorry,on my post I've added the wrong link. This is the correct one that I have fixed some minutes ago :

https://www.reddit.com/r/freebsd/comments/10ysy6m/how_to_install_on_freebsd_131release_a_different/

as you can see I've used my own thread posted on Reddit and based on your previous instructions for installing the nvidia driver version 525.78.01 and it is working now.

The problem is the resolution of my screen is stuck to 1024x768.

If you want to get back to 525 series, resign from using graphics/nvidia-drm-*-kmod and switch back to x11/nvidia-driver alone (or with x11/linux-nvidia-libs).

Which one of these libs should I install ?

Code:

# pkg search linux-nvidia-libs

linux-nvidia-libs-570.124.04   NVidia graphics libraries and programs (Linux version)
linux-nvidia-libs-304-304.137  NVidia graphics libraries and programs (Linux version)
linux-nvidia-libs-340-340.108  NVidia graphics libraries and programs (Linux version)
linux-nvidia-libs-390-390.154  NVidia graphics libraries and programs (Linux version)
linux-nvidia-libs-470-470.161.03 NVidia graphics libraries and programs (Linux version)

the nvidia-driver alone is not enough. I can't work with a resolution of 1024x768.

525.78.01
525.85.05
525.89.02
525.105.17
525.116.03
525.116.04

Picked from commented out NVIDIA_OVERRIDE_VERSION= lines in my /etc/make.conf.

Sorry I didn't understand how to increase the resolution of my screen and / or how to install the missing component that can do this. I think that installing the proper nvidia-drm-kmod version does it. Am I wrong ? If yes,I didn't understand how to do this. Please elaborate more.

ZioMario · May 4, 2025

Code:

# pkg install nvidia-drm-510-kmod

Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
    nvidia-drm-510-kmod: 570.124.04.1304000_1

Number of packages to be installed: 1

Proceed with this action? [y/N]: y
[1/1] Installing nvidia-drm-510-kmod-570.124.04.1304000_1...
[1/1] Extracting nvidia-drm-510-kmod-570.124.04.1304000_1: 100%
=====
Message from nvidia-drm-510-kmod-570.124.04.1304000_1:

--
Modesetting must be enabled to use nvidia-drm.ko for graphics. This can be done by setting the modeset sysctl, the equivalent of the modeset kernel parameter on Linux.

hw.nvidiadrm.modeset=1

This must be set before loading nvdidia-drm.ko, most easily done by placing the above in /boot/loader.conf.

# kldload nvidia-drm

kldload: an error occurred while loading module nvidia-drm. Please check dmesg(8) for more details.

DMESG :

KLD nvidia-drm.ko: depends on nvidia_modeset - not available or version mismatch
linker_load_file: /boot/modules/nvidia-drm.ko - unsupported file type

T-Aoki · May 4, 2025

DO NOT ATTEMPT TO INSTALL ANY OF graphics/nvidia-drm-*-kmod IF YOU NEED NVIDIA FREEBSD DRVER PRIOR TO 550 SERIES!!!!!!!!!!

IT IS IMPOSSIBLE (NO REQUIRED COMPONENTS ARE PROVIDED)!

First you SHALL do (read wording "SHALL" just as in ISO/IEC standards. No more or less intentions here!) is to deinstall all DRM drivers, as you want 510 series of driver to be compatible with Linux side you mentined in the thread.

Then, install x11/nvidia-driver (and optionally x11/linux-nvidia-libs if you need) 510.60.02 as your error message in Reddit indicates you need the exact version by overriding.

Keep in mind you need to keep sync with x11/nvidia-driver and x11/linux-nvidia-libs at the exact same version.

And the 510 indicates nvidia driver's major version, not LinuxKPI version!
graphics/nvidia-drm-*-kmod require distfiles from both x11/nvidia-driver and graphics/drm-*-kmod, and numbers in "*" here indicates which Linux kernel version (510 for 5.10[.*], 515 for 5.15[.*], 61 for 6.1[.*] and 66 for 6.6[.*]) to be emulated (and required by DRM drivers), completely unrelated with nvidia-driver major version.

And forget about x11/nvidia-driver-[304|340|390|470] and x11/linux-nvidia-libs-[304|340|390|470] for now, as they just overriding version.
None of them are for (nvidia) 510 series of drivers.
Note that the mechanism you're using for overriding version is primarily introduced to support these legacy versions of drivers.

For screen resolution, you would need configurations in your /etc/X11/xorg.conf. x11/nvidia-xconfig would help you doing so at least providing skeleton for the configuration. Don't forget to backup current version before invoking it. Not 510 series (currently 570 series to work on upgrading to 570.144), but my ThinkPad P52 with nvidia Quadro P1000 and 4k screen works with it, with and without graphics/nvidia-drm-61-kmod.

For example, my /etc/X11/xorg.conf (now actually renamed not to be picked as I'm on graphics/nvidia-drm-61-kmod active) has screen config like below.

Code:

Section "ServerLayout"
    Identifier     "PCI"
    Screen      0  "Screen PCI" 0 0
    Inactive       "InactiveDevice1"
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
    Option         "AIGLX" "True"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Sharp"
    ModelName      "DP-2"
    HorizSync       47.2 - 56.6
    VertRefresh     50.0 - 60.0
    ModeLine       "3840x2160_60.00" 712.75 3840 4160 4576 5312 2160 2163 2168 2237 -hsync +vsync
EndSection

Section "Screen"
    Identifier     "Screen PCI"
    Device         "NV PCI"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "metamodes" "nvidia-auto-select +0+0"
    Option         "RegistryDwords" "EnableBrightnessControl=1"
    Option         "UseEdidDpi" "FALSE"
    Option         "DPI" "284 x 284"
    Option         "AddARGBGLXVisuals" "True"
    Option         "UBB" "True"
    Option         "RenderAccel" "True"
    Option         "AllowGLXWithComposite" "True"
    Option         "AllowEmptyInitialConfiguration" "True"
    SubSection     "Display"
#        Modes       "3840x2160_60.00" "1280x1024" "1024x768" "800x600" "640x480"
        Depth       24
    EndSubSection
EndSection

Note that Modules, Inputs and GPU-related part etc. are ommitted.
ServerLayout section defines which screen config to use in Screen line,
Screen section includes which Monitor config to use in Monitor line,
And Monitor section defines the capabilities of the monitor connected.

ZioMario · May 4, 2025

Then, install x11/nvidia-driver (and optionally x11/linux-nvidia-libs if you need) 510.60.02 as your error message in Reddit indicates you need the exact version by overriding.

The driver that I have installed is 525.78.01 ; I don't have the same version of the linux-nvidia-libs from the packages. Maybe I should install the version I want from ports,modifying the Makefile like below ?

Code:

marietto# pwd   
/usr/ports

marietto# cd x11

marietto# cd linux-nvidia-libs

marietto# ls
Makefile    distinfo    pkg-descr    pkg-plist

marietto# nano Makefile 

DISTVERSION?= 525.78.01

T-Aoki · May 4, 2025

One thing to note.
Now pruning supports for intermediate versions of x11/nvidia-driver and x11/linux-nvidia-libs is proposed.
The first proposal was to prune on upgrading for (just ongoing) 570.114, but I've counter-proposed not to prune now but posting headsup to at least freebsd-ports ML with timeout (2 months or so, I imagine), then, prune on next Production branch upgrades.
If any replies to the headup that clarifying the use-case and valid reason why it's needed are posted, we'll consider keeping the support for the specific versions only (my current intention, though).

Anyway, it's still in review at D49982 and the final decision whether pruning now or after headsup is not yet made. And fix related with GSP firmware support at review D50053 needs to land first.

T-Aoki · May 4, 2025

ZioMario said:
The driver that I have installed is 525.78.01 ; I don't have the same version of the linux-nvidia-libs from the packages. Maybe I should install the version I want from ports,modifying the Makefile like below ?

Exactly. Overriding version is the method for building from ports, not at all pre-built official pkg.

Make 100% sure your src tree (usually /usr/src unless you've intentionally overridden) and your running kernel is at exact the same commit, always kept in sync except while building kernel through following restart.
x11/nvidia-driver itself is known to be "relatively" robust with the mis-match, but anything using LinuxKPI including graphics/drm-*-kmod, graphics/nvidia-drm-*-kmod are known to be quite, quite and quite fragile and usually mandates exact matching of the commit state.

ZioMario · May 4, 2025

Code:

===> Fetching all distfiles required by linux-nvidia-libs-525.78.01 for building

marietto:/usr/ports/x11/linux-nvidia-libs # make

===>  License NVIDIA accepted by the user
===>   linux-nvidia-libs-525.78.01 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by linux-nvidia-libs-525.78.01 for building
===>  Extracting for linux-nvidia-libs-525.78.01
=> SHA256 Checksum OK for NVIDIA-Linux-x86_64-525.78.01.run.
===>  Patching for linux-nvidia-libs-525.78.01
===>  Configuring for linux-nvidia-libs-525.78.01
===>  Staging for linux-nvidia-libs-525.78.01
===>   linux-nvidia-libs-525.78.01 depends on package: linux-c7-xorg-libs>=7.7 - found
===>   Generating temporary packing list
/usr/bin/sed -i.bak -E '\,/compat/linux,!{/libvdpau(_trace)?\.so/d;}'  /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/usr/bin/sed -i.bak -e '\,/compat/linux,!{/libXvMCNVIDIA/d;}' /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/usr/bin/sed -i.bak '\,/compat/linux,!s/libEGL\.so\.525.78.01/libEGL.so/'  /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/usr/bin/sed -i.bak -E '\,/compat/linux,!{/(EGL_nvidia\.so|GLESv[12](_CM)?(\.so\.[0-9]+\.[0-9]+))$/d;}'  /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/usr/bin/sed -i.bak -e '/tls\/libnvidia-tls\.so/d' /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/usr/bin/sed -i.bak -E -e '\,/compat/linux/usr/share/egl/egl_external_platform.d/20_nvidia_xcb.json,d' -e '\,/compat/linux/usr/share/egl/egl_external_platform.d/20_nvidia_xlib.json,d' -e '\,/compat/linux.*libnvidia-fatbinaryloader\.so,d' -e '\,/compat/linux,s/(libnvidia-egl-wayland\.so)\.X/\1.1.1.9/' -e '\,/compat/linux.*libnvidia-ifr,d' -e '\,/compat/linux.*libXvMCNVIDIA,d' -e '\,/compat/linux.*monitoring\.conf,d' -e '\,/compat/linux.*pci\.ids,d' -e '\,/compat/linux.*libnvidia-cbl\.so,d' -e '\,/compat/linux.*libnvidia-gpucomp\.so,d' -e '\,/compat/linux.*libnvidia-(egl-xcb|egl-xlib),d' -e '\,/compat/linux.*libnvoptix\.so,d' -e '\,/compat/linux.*libnvidia-(allocator|egl-gbm|ngx|nvvm|opticalflow),d' -e '\,/compat/linux.*libEGL\.so,d' -e '\,/compat/linux.*libGL\.so,d' -e '\,/compat/linux.*libGLESv(1_CM|2)\.so,d' -e '\,/compat/linux.*libGLX\.so,d' -e '\,/compat/linux.*libGLdispatch\.so,d' -e '\,/compat/linux.*libOpenGL\.so,d' /usr/ports/x11/linux-nvidia-libs/work/.PLIST.mktmp
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/etc/OpenCL/vendors
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia.icd /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/etc/OpenCL/vendors
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/glvnd/egl_vendor.d
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/10_nvidia.json /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/glvnd/egl_vendor.d
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/egl/egl_external_platform.d
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/10_nvidia_wayland.json /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/egl/egl_external_platform.d
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/nvidia
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-application-profiles-525.78.01-key-documentation /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/nvidia
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/nvidia
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-application-profiles-525.78.01-rc /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/nvidia
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/etc/vulkan/icd.d
install  -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia_icd.json /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/etc/vulkan/icd.d
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-bug-report.sh  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-debugdump /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-settings /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-smi /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-xconfig /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-cuda-mps-control /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-cuda-mps-server /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
install  -s -m 555 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-persistenced /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/bin
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
install  -m 444 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-settings.1.gz  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
install  -m 444 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-smi.1.gz  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
install  -m 444 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-xconfig.1.gz  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
install  -m 444 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-cuda-mps-control.1.gz  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
install  -m 444 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/nvidia-persistenced.1.gz  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/share/man/man1
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/tls
/bin/mkdir -p /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/vdpau
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libcuda.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libcuda.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvcuvid.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvcuvid.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-cfg.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-cfg.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-glcore.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-glcore.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-ml.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-ml.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-opencl.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-opencl.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-tls.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-tls.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libvdpau_nvidia.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libvdpau_nvidia.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libEGL_nvidia.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libEGL_nvidia.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libGLESv1_CM_nvidia.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libGLESv1_CM_nvidia.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libGLESv2_nvidia.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libGLESv2_nvidia.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libGLX_nvidia.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libGLX_nvidia.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-ptxjitcompiler.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-ptxjitcompiler.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-compiler.so.525.78.01  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-compiler.so.525.78.01
install  -s -m 0644 /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-egl-wayland.so.1.1.9  /usr/ports/x11/linux-nvidia-libs/work/stage/compat/linux/usr/lib64/libnvidia-egl-wayland.so.1.1.9
install: /usr/ports/x11/linux-nvidia-libs/work/NVIDIA-Linux-x86_64-525.78.01/libnvidia-egl-wayland.so.1.1.9: No such file or directory
*** Error code 71

Stop.
make: stopped in /usr/ports/x11/linux-nvidia-libs

T-Aoki · May 4, 2025

Not sure why I've not bitten by it when tested, but try below.

Download attached to any directory you want,
change directory to /usr/ports
and apply it with patch -p1 -i /directory/where/you/saved/the/patch/hotfix_nvidia_525_78_01.diff.txt.

You'll need to su (or sudo / doas) as root to apply.

ZioMario · May 4, 2025

With and without your xorg.conf file and without installing the nvidia-drm-kmod-570.124.04, Xorg (and nvidia-xconfig) are unabled to detect the screen. Should I install nvidia-drm-kmod-570.124.04 ? BTW I know for sure that the 570 is not compatible with the old nvidia driver that I'm running (NVIDIA-SMI 525.78.01 Driver Version: 525.78.01) ; what to do now ? This is the xorg.conf file generated by nvidia-xconfig :

Code:

# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig:  version 525.116.04

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0"
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
EndSection

Section "Files"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/sysmouse"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "keyboard"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Unknown"
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

ZioMario · May 4, 2025

Worked,adding this parameter to xorg.conf :

Code:

BusID          "PCI:1:0:0"

Very thanks. Now I can continue to work on the configuration of ComfyUI.

ZioMario · May 4, 2025

ZioMario · May 4, 2025

# /compat/ubuntu/usr/bin/nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
NVML library version: 570.124

# /mnt/da3p2/Backup/compat/noble/usr/bin/nvidia-smi
Failed to initialize NVML: Driver/library version mismatch

T-Aoki · May 5, 2025

ZioMario said:
Worked,adding this parameter to xorg.conf :

Code:

BusID "PCI:1:0:0"

Very thanks. Now I can continue to work on the configuration of ComfyUI.

The BusID would vary especially when you have multiple GPUs. At least differs between GPUs, IIUC. Otherwise drivers cannot attach per-GPU basis.

So you'd better having a habit to check BusID of (each, if multiple) GPU(s) via dmesg, pciconf and so on. If there's only one "display" class device (PCIe video cards, possibly including OCuLink, too), it would be auto-detected.
But if there's multiple GPUs, would need specifying each of them manually.

ZioMario · May 7, 2025

Ok. This is the second part of my experiment.

I've installed the nvidia driver and the linux-nvidia-libs vers. 565.77 and I started the miniconda installer to be able to install PyTorch , Stable Diffusion and / or ComfyUI on FreeBSD using the Linuxulator. The commands that I have issued,according with this tutorial :

GitHub - verm/freebsd-stable-diffusion: Stable Diffusion on FreeBSD with CUDA support

Stable Diffusion on FreeBSD with CUDA support. Contribute to verm/freebsd-stable-diffusion development by creating an account on GitHub.

github.com

are the following :

Code:

marietto@marietto:/usr/home/marietto $ pkg install libc6-shim linux-miniconda-installer

marietto@marietto:/usr/home/marietto $ conda-shell

Starting Linux bash shell to ensure that Conda packages are fully functional.

Note that uname will return Linux rather than FreeBSD and some other aspects
of the environment will also indicate Linux.

To activate conda, run the "eval" command indicated during the conda
installation, e.g.

eval ""

Type "exit" or "Ctrl+d" to return to your FreeBSD shell.


I have no name!@marietto:/usr/home/marietto$ source conda.sh

I have no name!@marietto:/usr/home/marietto$ conda activate

(base) I have no name!@marietto:/usr/home/marietto$ conda create --name pytorch python=3.10

(base) I have no name!@marietto:/usr/home/marietto$ conda activate pytorch

(pytorch) I have no name!@marietto:/usr/home/marietto$ pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu127

unfortunately it does not work :

Code:

(pytorch) I have no name!@marietto:/usr/home/marietto$ LD_PRELOAD="./dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.is_available())'

/home/marietto/miniconda3/envs/pytorch/lib/python3.10/site-packages/torch/cuda/__init__.py:181: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error?

Error 304: OS call failed or operation not supported on this OS (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0

False

I don't understand the reason,because the nvidia drivers and libs are configured correctly :

Code:

(pytorch) I have no name!@marietto:/usr/home/marietto$ nvidia-smi

Wed May  7 13:39:19 2025   
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.77                 Driver Version: 565.77         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1060 3GB    Off |   00000000:01:00.0  On |                  N/A |
| 53%   32C    P8             11W /  120W |     333MiB /   3072MiB |      3%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 2080 Ti     Off |   00000000:02:00.0 Off |                  N/A |
| 31%   35C    P8             21W /  250W |       2MiB /  11264MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                     
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      4127      G   /usr/local/libexec/Xorg                       177MiB |
|    0   N/A  N/A      4165      G   xfwm4                                           2MiB |
|    0   N/A  N/A      4247      G   /usr/local/lib/firefox/firefox                149MiB |
+-----------------------------------------------------------------------------------------+

In my previous experiment,I've installed the nvidia driver and the linux-nvidia-libs vers. 525.78.01 and "torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113" and it worked like a charme :

Code:

(pytorch) I have no name!@marietto:/usr/home/marietto$ LD_PRELOAD="/compat/dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.is_available())'

True

(pytorch) I have no name!@marietto:/usr/home/marietto$ LD_PRELOAD="/compat/dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.get_device_name(0))'

NVIDIA GeForce RTX 2080 Ti

Can you guess why ?

T-Aoki · May 7, 2025

I'm not using any Linux stuff you mentioned, so this is quite rough and stupid prediction.

Have you installed the same version of x11/linux-nvidia-libs as x11/nvidia-driver you've installed (here, I guess 565.77)? If no, replace installed version of x11/linux-nvidia-libs with the same version as x11/nvidia-driver via ports.

Are you sure you rebuilt and reinstalled everything you've built / linked against previous x11/nvidia-driver and/or x11/linux-nvidia-libs? Maybe dummy-uvm.so would be the one, but not sure as I have nothing to test with in hand.

Recently default Linuxulator userlands switched from c7 to rl9 for amd64. Are you sure you've not mangled both?

ZioMario · May 7, 2025

I think,yes :

Code:

marietto@marietto:/usr/ports/x11/nvidia-driver $ nano Makefile :

PORTNAME?=      nvidia-driver
#DISTVERSION?=  ${NVIDIA_DISTVERSION}
DISTVERSION?=   565.77
# Always try to set PORTREVISION as it can be overridden by the slave ports
#PORTREVISION?= 2
PORTREVISION?=  0
CATEGORIES=     x11
MASTER_SITES=   NVIDIA/XFree86/FreeBSD-${ARCH_SUFX}/${DISTVERSION}
DISTNAME=       NVIDIA-FreeBSD-${ARCH_SUFX}-${DISTVERSION}
DISTFILES=      ${DISTNAME}${EXTRACT_SUFX}

marietto@marietto:/usr/ports/x11/linux-nvidia-libs $ nano Makefile

PORTNAME=       nvidia-libs
#DISTVERSION?=  ${NVIDIA_DISTVERSION}
DISTVERSION?=   565.77
# Always try to set PORTREVISION as it can be overridden by the slave ports
PORTREVISION?=  0
CATEGORIES=     x11 linux
MASTER_SITES=   NVIDIA/XFree86/Linux-${ARCH_SUFX}/${DISTVERSION}/
DISTNAME=       NVIDIA-Linux-${ARCH_SUFX}-${DISTVERSION}
EXTRACT_SUFX=   .run
PKGNAMEPREFIX=  linux-

ZioMario · May 7, 2025

Maybe dummy-uvm.so would be the one, but not sure as I have nothing to test with in hand....

this is a very interesting point. Anyway I have no idea about how to recompile it...tutorial does not explain the procedure so well...

This is what Verm says to do :

Next get uvm_ioctl_override.c.

# fetch https://gist.githubusercontent.com/...af405c95c457ce4afc85c608/uvm_ioctl_override.c

Build.

# /compat/linux/bin/cc --sysroot=/compat/linux -m64 -std=c99 -Wall -ldl -fPIC -shared -o dummy-uvm.so uvm_ioctl_override.c

Code:

marietto@marietto:/mnt/da2p2/Backup/compat/noble/bin $ cc --sysroot=/mnt/da2p2/Backup/compat/noble -m64 -std=c99 -Wall -ldl -fPIC -shared -o /mnt/da2p2/Backup/compat/dummy-uvm.so /mnt/da2p2/Backup/uvm_ioctl_override.c

In file included from /mnt/da2p2/Backup/uvm_ioctl_override.c:3:
In file included from /mnt/da2p2/Backup/compat/noble/usr/include/dlfcn.h:22:
In file included from /mnt/da2p2/Backup/compat/noble/usr/include/features.h:394:
/mnt/da2p2/Backup/compat/noble/usr/include/features-time64.h:20:10: fatal error: 'bits/wordsize.h' file not found
   20 | #include <bits/wordsize.h>
      |          ^~~~~~~~~~~~~~~~~
1 error generated.

marietto@marietto:/mnt/da2p2/Backup/compat/noble/bin $ find .. -name wordsize.h
../include/x86_64-linux-gnu/bits/wordsize.h

the missing file seems to be in the right place,bro...

NapoleonWils0n · May 7, 2025

i spent several weeks trying to get stable diffusion and comfy-ui to work without success
if you have seen my threads on getting Davinci Resolve to work with the 535 driver in a jail you will know how many permutations i had to go through

or my thread on issues with python torch on freebsd

i did explain many times that cuda in a jail or with the linuxulator has been broken since the 535 driver
i know this because i had davinci resolve installed in both an Ubuntu and Rocky Linux jail, and using the linuxulator

and after the 535 driver cuda does not show up as an option in resolve
also whipser install in a python venv does not work with cuda either

the nvidia 570 driver didnt work with my NVIDIA GeForce GTX 1650
it just crashed the system trying to play a video with firefox or mpv

there arent that many option to set with the driver and i tried they all in another thread with T-Aoki
and none of them worked, so i had to switch my main machine ( dell xps 15 2019 ) to nixos

( still have freebsd installed on my old macbook air 2011 )

i am using nixos to run stable diffusion and comfy-ui

the issue isnt just related to python or conda venv's
linux in a jail or using the linuxulator doesnt work with drivers after 535

it could be related to the driver or the dummy-uvm.so or the linux sys calls
but either way there are too many parts that all have to work together that can easily break on Freebsd

you need to have freebsd working with the drivers and the shim, and then install linux

recent version of Ubuntu in a jail or the linuxulator require constant fixes due to systemd breaking apt
in a jail you have to manually install the exact same driver as on freebsd and then use LD_PRELOAD to load the dummy-uvm.so

comfy-ui custom nodes are all built with python, so its not just python torch torchvision torchaudio you need to install
there are loads of other python packages that are required in addition you may run into issue actually try to use the models on freebsd

i have been through all the different permutations, including install the freebsd python torch packages
and using python venv with allow site-packages to try and use the freebsd torch packages
didnt work

even if you actually managed to get both stable diffusion and comfy-ui installed
it wouldnt be stable and would break too easily

so short answer is use the right tool for the job
which in this case isnt Freebsd, you need to use Linux on bare metal

building the dummy-uvm.so instructions are in the file

Code:

// pkg install linux-c7-devtools
// /compat/linux/bin/cc --sysroot=/compat/linux -m64 -std=c99 -Wall -ldl -fPIC -shared -o dummy-uvm.so uvm_ioctl_override.c
// env LD_PRELOAD=$PWD/dummy-uvm.so

Code:

#define _GNU_SOURCE

#include <dlfcn.h>
#include <fcntl.h>
#include <string.h>
#include <stdarg.h>
#include <stdint.h>

// pkg install linux-c7-devtools
// /compat/linux/bin/cc --sysroot=/compat/linux -m64 -std=c99 -Wall -ldl -fPIC -shared -o dummy-uvm.so uvm_ioctl_override.c
// env LD_PRELOAD=$PWD/dummy-uvm.so <cmd>

#define NV_UVM_INITIALIZE   0x30000001
#define NV_UVM_DEINITIALIZE 0x30000002

#define NV_ERR_NOT_SUPPORTED 0x56

struct NvUvmInitParams
{
  uint64_t flags __attribute__((aligned(8)));
  uint32_t status;
};

int (*libc_ioctl)(int fd, unsigned long request, ...) = NULL;

int ioctl(int fd, unsigned long request, ...) {

  if (!libc_ioctl) {
    libc_ioctl = dlsym(RTLD_NEXT, "ioctl");
  }

  va_list _args_;
  va_start(_args_, request);
  void* data = va_arg(_args_, void*);
  va_end(_args_);

  if (request == NV_UVM_INITIALIZE) {
    struct NvUvmInitParams* params = (struct NvUvmInitParams*)data;
    params->status = NV_ERR_NOT_SUPPORTED;
    return 0;
  }

  if (request == NV_UVM_DEINITIALIZE) {
    return 0;
  }

  return libc_ioctl(fd, request, data);
}

int (*libc_open)(const char* path, int flags, ...) = NULL;

int open(const char* path, int flags, ...) {

  if (!libc_open) { libc_open = dlsym(RTLD_NEXT, "open"); }

  mode_t mode = 0;

  va_list _args_;
  va_start(_args_, flags);

  if (flags & O_CREAT) {
    mode = va_arg(_args_, int);
  }

  va_end(_args_);

  if (strcmp("/dev/nvidia-uvm", path) == 0) {
    return libc_open("/dev/null", flags, mode);
  }

  return libc_open(path, flags, mode); 
}

ZioMario · May 7, 2025

Ubuntu in a jail :

Ubuntu in the Jar :

View: https://www.youtube.com/watch?v=boanuwUMNNQ

ZioMario · May 7, 2025

building the dummy-uvm.so instructions are in the file

// pkg install linux-c7-devtools

this is not valid anymore. I'm playing with different permutations since Jason gave me an upgraded version of the miniconda installer...now it works with linux-rl9....

How to install the nvidia-drm-kmod for the driver nvidia 525.78.01 on FreeBSD 14.2 to be able to run ComfyUI.

You should already know how to override versions forward and backward, as you posted the very next one. And another.​

Attachments

You should already know how to override versions forward and backward, as you posted the very next one. And another.