general/other Cuda working on Freebsd using Podman - support thread

NapoleonWils0n · May 29, 2026

Run Linux command line and gui applications on FreeBSD with full Nvidia Cuda hardware acceleration.

I have also posted this in the how to section
But that has comments disabled by default from memory

mods i created another thread so people post comments
and you dont have to approve them in the how to section, just trying to save you some work

Note the video does have chapters so you can jump to sections

GitHub - NapoleonWils0n/freebsd-cuda: Run Linux command line and gui applications on FreeBSD with full Nvidia Cuda hardware acceleration.

Run Linux command line and gui applications on FreeBSD with full Nvidia Cuda hardware acceleration. - NapoleonWils0n/freebsd-cuda

github.com

Mounts the Linuxulator libraries from Freebsd into the Podman containers, so you dont need to install the Nvidia driver into the container

Wayland/X11 and PulseAudio Pass-through

Displays graphics and routes audio via native FreeBSD sockets mounted inside the Podman container

Hardware acclerated video

Persistent storage for files so settings are retained when stopping containers

Directory mounted from Freebsd into the Podman container for file access

GUI Applications launched with desktop entries on Freebsd
Pre-configured Applications

Firefox with hardware accelerated video and Widevine DRM support

ffmpeg with hardware-accelerated nvenc encoding

WhisperX audio transcription with Cuda acceleration

Ollama using Cuda

Root dotfiles

GitHub - NapoleonWils0n/freebsd-root-xps: freebsd root dotfiles for dell xps 15 2019

freebsd root dotfiles for dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-root-xps development by creating an account on GitHub.

github.com

dotfile

GitHub - NapoleonWils0n/freebsd-dotfiles-xps: freebsd dotfiles dell xps 15 2019

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.

github.com

bin

GitHub - NapoleonWils0n/freebsd-bin-xps: freebsd-bin-xps

freebsd-bin-xps. Contribute to NapoleonWils0n/freebsd-bin-xps development by creating an account on GitHub.

github.com

NapoleonWils0n · May 29, 2026

The project is built for Wayland and X11

Im running Wayland so havent been able to test it on X11
but it should work

So we need some X11 Guinea pigs to test it out

grandpa · Sep 9, 2026

So. Guinea pig here.

I did the below steps. I have put a *** for the the steps I have changed.

Ollama runs but only with CPU. Card is rtx3060, Tried CUDA_VISIBLE_DEVICES=0 ollama run llama3.1:8b but still CPU only

I would like to start looking at doing llama-cpp instead of ollama but it would be nice to verify that ollama works first.

I have deliberately skipped everything except x11 and ollama. The strangest part is that there is log for the ollama install?

 *** python312

*** sudo pkg install podman-suite py312-podman-compose



sudo mount -t fdescfs fdesc /dev/fd

sudo mount -t procfs proc /proc



*** added proc while editing fstab

*** sudo vi /etc/fstab



fdesc   /dev/fd         fdescfs         rw      0       0

*** proc    /proc            procfs         rw      0       0



sudo sysrc linux_enable="YES"

sudo service linux start

sudo kldload linux64

sudo sysrc kld_list+="linux64"

service pf enable

service pf start



*** it's pf_enable

*** sudo sysrc 'pf_enable=YES'  XXXXX



kldload pf

vi /etc/pf.conf



*** rewrote the pf.conf 

*** pf.conf changed

#=========================================================================#

# MINIMAL PF.CONF FOR PODMAN (STRICT RULES ORDER)

#=========================================================================#



# 1. MACROS & TABLES

ext_if = "lagg0"

table <cni-nat>



# 2. OPTIONS

set skip on { lo0, cni-podman0 }



# 3. TRANSLATION & REDIRECTION (NAT & NAT Anchors Only)

nat-anchor "cni-rdr/*"

rdr-anchor "cni-rdr/*"



nat on $ext_if inet from <cni-nat> to any -> ($ext_if)



# 4. FILTERING RULES & ANCHORS (Must be at the very bottom)

anchor "cni/*"

pass all



sudo service pf restart

sudo zfs create -o mountpoint=/var/db/containers zroot/containers

sudo sysctl net.pf.filter_local=1

sudo vi /etc/sysctl.conf

#=========================================================================#

# podman

#=========================================================================#



net.pf.filter_local=1



sudo sysrc podman_enable=YES

sudo vi /usr/local/etc/containers/registries.conf

unqualified-search-registries = ["docker.io", "quay.io", "ghcr.io"]



[[registry]]

location = "docker.io"



[[registry]]

location = "quay.io"



[[registry]]

location = "ghcr.io"



*** added devfs ruleset for container 

*** sudo vi /usr/local/etc/containers/containers.conf

*** [containers]

*** jail_devfs_ruleset = "5"



sudo zfs create zroot/home/user/podman

sudo chown -R user:group /home/user/podman



*** This breaks on boot with zfs error

*** DO NOT DO THIS

*** sudo vi /etc/rc.conf

*** devfs_system_ruleset="localrules"



*** changed devfs rules

*** sudo vi /etc/devfs.rules

[localrules=5]

add include $devfsrules_hide_all

add include $devfsrules_unhide_basic

add include $devfsrules_unhide_login



add path 'dri*' unhide

add path 'dri/*' mode 0660 group video

add path 'drm*' unhide

add path 'drm/*' mode 0660 group video

add path 'nvidia*' unhide

add path 'nvidia*' mode 0660 group video



add path 'input/*' mode 0660 group video

add path 'input/event*' mode 0660 group video



add path "nvidia-uvm*" unhide

add path "nvidia-uvm*" mode 0666



*** added to see zfs

*** # FIX FOR PODMAN ZFS STORAGE ENGINE:

*** add path 'zfs' unhide

*** add path 'zfs' mode 0660 group group



sudo podman run --rm docker.io/dougrabson/hello

- success -



cd ~/podman

git clone https://github.com/NapoleonWils0n/freebsd-cuda.git

cd ~/podman/freebsd-cuda/base-build/base-image/x11

sudo podman build \

--network=host \

--os linux \

--arch amd64 \

--build-arg HOST_UID="$(id -u)" \

--build-arg HOST_USER="${USER}" \

-t localhost/base-image .

- success - 



sudo zfs create zroot/home/user/.ollama

sudo chown -R user:group /home/user/.ollama

cd ~/podman/freebsd-cuda/containers/ai/ollama

make Makefile

make

- success -



sudo podman build \

--network=host \

--os linux \

--arch amd64 \

--build-arg HOST_USER="${USER}" \

--build-arg OLLAMA_VERSION="v0.33.3" \

-t localhost/ollama .

- success - 



sudo podman-compose logs -f

- gives no result at all - 



sudo podman-compose up -d

sudo podman exec -it -u ${USER} ollama /bin/bash

- runs fine but is CPU only -

/grandpa

NapoleonWils0n · Sep 9, 2026

Hi Mate

all the steps look correct
good job getting podman installed and the project

can you check a couple of things to debug the issue

i can see you ran make with the makefile

if you run ls -al in the ollama directory

Code:

ls -la

Does it show the .env and output like this
with the

Code:

[i] Yes Master ? cd ~/podman/freebsd-cuda/containers/ai/ollama
[djwilcox@pollux ~/podman/freebsd-cuda/containers/ai/ollama](master =)
[i] Yes Master ? ls -la
total 32
drwxr-xr-x  3 djwilcox djwilcox    7  6 Sep 16:33 .
drwxr-xr-x  3 djwilcox djwilcox    3 27 May 13:25 ..
-rw-r--r--  1 djwilcox djwilcox 1586  6 Sep 16:33 .env
-rw-r--r--  1 djwilcox djwilcox 2071 27 May 17:02 Dockerfile
-rw-r--r--  1 djwilcox djwilcox 3191 19 Jun 14:01 Makefile
drwxr-xr-x  2 djwilcox djwilcox    4 19 Jun 14:40 bin
-rw-r--r--  1 djwilcox djwilcox 9662 19 Jun 14:02 compose.yaml

on this step

Code:

 sudo chown -R user:group /home/user/.ollama

did you replace user:group with your username
and /home/user withe your username

Code:

doas chown -R djwilcox:djwilcox /home/djwilcox/.ollama

1) start the container

Code:

sudo podman-compose up -d

2 ) enter the container

Code:

sudo podman exec -it -u ${USER} ollama /bin/bash

3) run ollama serve

Code:

ollama serve

4) in a new terminal or tmux pane enter the container again

Code:

cd ~/podman/freebsd-cuda/containers/ai/ollama
sudo podman exec -it -u ${USER} ollama /bin/bash

list the ollama models

Code:

[i] Yes Master ? cd ~/podman/freebsd-cuda/containers/ai/ollama
[djwilcox@pollux ~/podman/freebsd-cuda/containers/ai/ollama](master =)
[i] Yes Master ? doas podman exec -it -u ${USER} ollama /bin/bash
bash-5.1$ ollama list
NAME                                ID              SIZE      MODIFIED
qwen2.5-coder:7b-instruct-q4_K_M    dae161e27b0e    4.7 GB    6 days ago
translategemma:4b                   c49d986b0764    3.3 GB    6 days ago
huihui_ai/llama3.2-abliterate:3b    2ffe056672b5    2.2 GB    2 weeks ago
gemma4-unsloth:latest               03a5e0cfa4f0    2.6 GB    2 months ago
gemma4-qat:latest                   43fa363e649f    3.3 GB    2 months ago
gemma4-bartowski:latest             d88dc450b698    3.5 GB    2 months ago
granite4.1:3b-q6_K                  3c142316f4cf    2.8 GB    4 months ago

run a model

Code:

ollama run qwen2.5-coder:7b-instruct-q4_K_M

5) in a new terminal or tmux pane enter the container again

Code:

cd ~/podman/freebsd-cuda/containers/ai/ollama
sudo podman exec -it -u ${USER} ollama /bin/bash

run nvidia-smi

Code:

nvidia-smi

the output should look like this

Code:

bash-5.1$ nvidia-smi
Wed Sep  9 12:09:42 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84                 Driver Version: 595.84         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1650        Off |   00000000:01:00.0 Off |                  N/A |
| N/A   53C    P8              2W /   50W |    2818MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A           78979      C   /usr/lib/ollama/llama-server           2812MiB |
+-----------------------------------------------------------------------------------------+
b

Note you will only Ollama listed when you run nvidia-smi in the container
not if you run nvidia-smi on the freebsd host

nvidia-smi run on the freebsd host

Code:

[i] Yes Master ? nvidia-smi
Wed Sep  9 13:11:53 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84                 Driver Version: 595.84         CUDA Version: N/A      |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1650        Off |   00000000:01:00.0 Off |                  N/A |
| N/A   51C    P8              2W /   50W |    2818MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
[

grandpa · Sep 9, 2026

Ignore this post.

/grandpa

grandpa · Sep 9, 2026

Ok, so this is how it looks.

 bash-5.1$ ollama serve

time=2026-09-09T14:31:39.533Z level=INFO source=routes.go:1955 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVI

CES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: LLAMA_ARG_FIT: LLAMA_ARG_FIT_TARGET: 

NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA

_GO_TEMPLATE:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_IGPU_ENABLE: OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_

TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_TRANSFER_STREAMS:4 OLLAM

A_MODELS:/home/user/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_OR

IGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:*

 https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* v

scode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:true ROCR_VISIBLE_DEVICES: http_proxy: https_proxy

: no_proxy:]"

 sudo podman exec -it -u ${USER} ollama /bin/bash

bash-5.1$ ollama run llama3.1:8b

>>> Send a message (/? for help)

 

bash-5.1$ nvidia-smi

bash: nvidia-smi: command not found

On the BSD side:

 nvidia-smi

Wed Sep  9 16:36:47 2026       

+-----------------------------------------------------------------------------------------+

| NVIDIA-SMI 595.99.02              Driver Version: 595.99.02      CUDA Version: N/A      |

+-----------------------------------------+------------------------+----------------------+

| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |

| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |

|                                         |                        |               MIG M. |

|=========================================+========================+======================|

|   0  NVIDIA GeForce RTX 3060        Off |   00000000:01:00.0  On |                  N/A |

|  0%   53C    P8             18W /  170W |     315MiB /  12288MiB |      0%      Default |

|                                         |                        |                  N/A |

+-----------------------------------------+------------------------+----------------------+



+-----------------------------------------------------------------------------------------+

| Processes:                                                                              |

|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |

|        ID   ID                                                               Usage      |

|=========================================================================================|

|    0   N/A  N/A            2047      G   /usr/local/libexec/Xorg                 147MiB |

|    0   N/A  N/A            2090      G   ...ibexec/xdg-desktop-portal-kde          2MiB |

|    0   N/A  N/A            2094      G   /usr/local/bin/kded6                      2MiB |

|    0   N/A  N/A            2097      G   /usr/local/bin/kwin_x11                  13MiB |

|    0   N/A  N/A            2103      G   /usr/local/bin/ksmserver                  2MiB |

|    0   N/A  N/A            2125      G   /usr/local/bin/kaccess                    2MiB |

|    0   N/A  N/A            2126      G   /usr/local/bin/plasmashell               25MiB |

|    0   N/A  N/A            2127      G   ...it-kde-authentication-agent-1          2MiB |

|    0   N/A  N/A            2130      G   .../lib/libexec/DiscoverNotifier          2MiB |

|    0   N/A  N/A            2141      G   /usr/local/bin/konsole                    2MiB |

|    0   N/A  N/A            2142      G   /usr/local/bin/konsole                    2MiB |

|    0   N/A  N/A            2143      G   /usr/local/bin/konsole                    2MiB |

|    0   N/A  N/A            2144      G   /usr/local/bin/konsole                    2MiB |

|    0   N/A  N/A            2149      G   /usr/local/bin/kate                       2MiB |

|    0   N/A  N/A            2199      G   ...rack-uuid=3190708988185955192         44MiB |

|    0   N/A  N/A            5089      G   /usr/local/bin/konsole                    2MiB |

+-----------------------------------------------------------------------------------------+

/grandpa

cracauer@ · Sep 9, 2026

I am curious what this CUDA test program outputs in podman. Attached as source and compiled with Rocky.

NapoleonWils0n · Sep 9, 2026

I am curious what this CUDA test program outputs in podman. Attached as source and compiled with Rocky.

Tested in a Rocky Linux Podman container and it works

Code:

bash-5.1$ ./hello-world-cuda2
Using device: NVIDIA GeForce GTX 1650

Hello, World! from thread 4 (block 1, thread 0)
Hello, World! from thread 5 (block 1, thread 1)
Hello, World! from thread 6 (block 1, thread 2)
Hello, World! from thread 7 (block 1, thread 3)
Hello, World! from thread 0 (block 0, thread 0)
Hello, World! from thread 1 (block 0, thread 1)
Hello, World! from thread 2 (block 0, thread 2)
Hello, World! from thread 3 (block 0, thread 3)

grandpa · Sep 9, 2026

NapoleonWils0n said:

if you run ls -al in the ollama directory

Code:

ls -la

Does it show the .env and output like this
with the

Code:

[i] Yes Master ? cd ~/podman/freebsd-cuda/containers/ai/ollama
[djwilcox@pollux ~/podman/freebsd-cuda/containers/ai/ollama](master =)
[i] Yes Master ? ls -la
total 32
drwxr-xr-x  3 djwilcox djwilcox    7  6 Sep 16:33 .
drwxr-xr-x  3 djwilcox djwilcox    3 27 May 13:25 ..
-rw-r--r--  1 djwilcox djwilcox 1586  6 Sep 16:33 .env
-rw-r--r--  1 djwilcox djwilcox 2071 27 May 17:02 Dockerfile
-rw-r--r--  1 djwilcox djwilcox 3191 19 Jun 14:01 Makefile
drwxr-xr-x  2 djwilcox djwilcox    4 19 Jun 14:40 bin
-rw-r--r--  1 djwilcox djwilcox 9662 19 Jun 14:02 compose.yaml

on this step

Code:

 sudo chown -R user:group /home/user/.ollama

did you replace user:group with your username
and /home/user withe your username

Code:

doas chown -R djwilcox:djwilcox /home/djwilcox/.ollama

Yes and yes.

/grandpa

NapoleonWils0n · Sep 9, 2026

that rules those out then

bash: nvidia-smi: command not found

looks like a devfs issue

you seem to be missing this in devfs.rules

Code:

[devfsrules_jail=4]
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide

podman containers are actually oci jails
and they use [devfsrules_jail=4]

this is my full devfs.rules

Code:

[localrules=5]
add path 'da*' mode 0660 group operator
add path 'dri/*' mode 0660 group video
add path 'drm/*' mode 0660 group video
add path 'input/*' mode 0660 group video
add path 'input/event*' mode 0660 group video
add path 'nvidia*' mode 0660 group video

[devfsrules_jail=4]
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide

grandpa · Sep 9, 2026

So when I got creative and added jail_devfs_ruleset to containers.conf I botched it? The purpose was to make only podman read the rules.

I'll remove the lines from containers.conf and try your devfs.

 sudo vi /usr/local/etc/containers/containers.conf

[containers]

jail_devfs_ruleset = "5"

 [localrules=5]

add include $devfsrules_hide_all

add include $devfsrules_unhide_basic

add include $devfsrules_unhide_login



add path 'dri*' unhide

add path 'dri/*' mode 0660 group video

add path 'drm*' unhide

add path 'drm/*' mode 0660 group video

add path 'nvidia*' unhide

add path 'nvidia*' mode 0660 group video



add path 'input/*' mode 0660 group video

add path 'input/event*' mode 0660 group video



add path "nvidia-uvm*" unhide

add path "nvidia-uvm*" mode 0666



# CRITICAL FIX FOR PODMAN ZFS STORAGE ENGINE:

add path 'zfs' unhide

add path 'zfs' mode 0660 group wheel

/grandpa

grandpa · Sep 9, 2026

So I commented the line jail_devfs_ruleset="5" in /usr/local/etc/containers/container.conf and changed /etc/devfs.rules to your default but still get

 bash-5.1$ nvidia-smi

bash: nvidia-smi: command not found

Mind you - I have NOT added devfs_system_ruleset="localrules" to /etc/rc.conf because if I do this pc boots to a blank screen (ie no sddm-greeter) and just sits there forcing me to boot single and do mount -u / && zfs mount -a to be able to remove the line in rc.conf.

/grandpa

NapoleonWils0n · Sep 9, 2026

hi mate

devfs.rules - FreeBSD 15.1-RELEASE and Ports.quarterly Manual Pages

man.freebsd.org

the devfs rules for the podman container
have to be under this section for it to work [devfsrules_jail=4]

and the localrules 5 is needed to set the permissions
so the user in the podman container which is set to use the video group can access the devides

Code:

[localrules=5]
add path 'da*' mode 0660 group operator
add path 'dri/*' mode 0660 group video
add path 'drm/*' mode 0660 group video
add path 'input/*' mode 0660 group video
add path 'input/event*' mode 0660 group video
add path 'nvidia*' mode 0660 group video

[devfsrules_jail=4]
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide

grandpa · Sep 9, 2026

NapoleonWils0n said:
hi mate

devfs.rules - FreeBSD 15.1-RELEASE and Ports.quarterly Manual Pages

man.freebsd.org

the devfs rules for the podman container
have to be under this section for it to work [devfsrules_jail=4]

and the localrules 5 is needed to set the permissions
so the user in the podman container which is set to use the video group can access the devides

Code:

[localrules=5] add path 'da*' mode 0660 group operator add path 'dri/*' mode 0660 group video add path 'drm/*' mode 0660 group video add path 'input/*' mode 0660 group video add path 'input/event*' mode 0660 group video add path 'nvidia*' mode 0660 group video [devfsrules_jail=4] add include $devfsrules_unhide_basic add include $devfsrules_unhide_login add path 'mixer*' unhide add path 'dsp*' unhide add path 'dri*' unhide add path 'drm*' unhide add path 'nvidia*' unhide add path 'speaker*' unhide

Yes, I have that in /etc/devfs.rules.

 more /etc/devfs.rules

[localrules=5]

add path 'da*' mode 0660 group operator

add path 'dri/*' mode 0660 group video

add path 'drm/*' mode 0660 group video

add path 'input/*' mode 0660 group video

add path 'input/event*' mode 0660 group video

add path 'nvidia*' mode 0660 group video



[devfsrules_jail=4]

add include $devfsrules_unhide_basic

add include $devfsrules_unhide_login

add path 'mixer*' unhide

add path 'dsp*' unhide

add path 'dri*' unhide

add path 'drm*' unhide

add path 'nvidia*' unhide

add path 'speaker*' unhide

As I mentioned I can't set devfs_system_ruleset="localrules".

Also podman logs outputs nothing.

Ollama runs, it's just not running on the graphics card, it's using the CPU... Chugging along at

 slot print_timing: id  0 | task 0 | n_gen =    202, tg =   3.80 t/s, tg_3s =   3.50 t/s

slot print_timing: id  0 | task 0 | n_gen =    213, tg =   3.78 t/s, tg_3s =   3.46 t/s

 

bash-5.1$ ollama run llama3.1:8b  

>>> Tell me about napoleonWils0nI couldn't find any information on a person named "napoleonWils0n." It's possible that 

this is a misspelling or a variation of a name that I'm not aware of.



However, I did find information on a few notable individuals with similar names:



* Napoleon Bonaparte (1769-1821): A French military leader and politician who rose to 

prominence during the French Revolution and became Emperor of the French.

* Wilson (a surname shared by several notable individuals, such as):

        + Woodrow Wilson (1856-1924): The 28th President of the United States, who served fr

om 

1913 to 1921 and was a key figure in World War I.

        + Brian Wilson (1942-present): An American musician, singer, and songwriter, best kn



known as the leader of the Beach Boys.

        + Nancy Wilson (1937-2018): An American jazz singer and guitarist.



If you could provide more context or clarify who you are looking for, I'll do my best 

to provide more information.



>>> Send a message (/? for help)

 nvidia-smi

Wed Sep  9 19:12:30 2026       

+-----------------------------------------------------------------------------------------+

| NVIDIA-SMI 595.99.02              Driver Version: 595.99.02      CUDA Version: N/A      |

+-----------------------------------------+------------------------+----------------------+

| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |

| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |

|                                         |                        |               MIG M. |

|=========================================+========================+======================|

|   0  NVIDIA GeForce RTX 3060        Off |   00000000:01:00.0  On |                  N/A |

|  0%   51C    P8             18W /  170W |     313MiB /  12288MiB |

Maybe nvidia-smi won't run in the container since a) it might not be in the base-build(?) or b) might need some component that is not in the base-build?

Maybe ollama pulls in pre-built libs that supercede the dummy-uvm?

Maybe I need to install all other parts of the project? Wrapper? All of them?

/grandpa

grandpa · Sep 9, 2026

Ok, so Rocky should have libcuda somewhere?

 ls /compat/linux/usr/lib64/libc*

/compat/linux/usr/lib64/libc.so.6

/compat/linux/usr/lib64/libc_malloc_debug.so.0

/compat/linux/usr/lib64/libcairo-gobject.so.2

/compat/linux/usr/lib64/libcairo-gobject.so.2.11704.0

/compat/linux/usr/lib64/libcairo-script-interpreter.so.2

/compat/linux/usr/lib64/libcairo-script-interpreter.so.2.11704.0

/compat/linux/usr/lib64/libcairo.so.2

/compat/linux/usr/lib64/libcairo.so.2.11704.0

/compat/linux/usr/lib64/libcap.so.2

/compat/linux/usr/lib64/libcap.so.2.48

/compat/linux/usr/lib64/libcom_err.so.2

/compat/linux/usr/lib64/libcom_err.so.2.1

/compat/linux/usr/lib64/libcrypt.so.2

/compat/linux/usr/lib64/libcrypt.so.2.0.0

/compat/linux/usr/lib64/libcrypto.so.3

/compat/linux/usr/lib64/libcrypto.so.3.5.1

/compat/linux/usr/lib64/libcups.so.2

/compat/linux/usr/lib64/libcupsimage.so.2

What am I missing here?

/grandpa

NapoleonWils0n · Sep 9, 2026

what are the permission on /dev/nvidia

Code:

ls -l /dev/nvidia*

the devfs.rules change the permissions to this

Code:

crw-rw----  1 root video 0xad  9 Sep 11:53 /dev/nvidia-modeset
crw-rw----  1 root video 0xab  9 Sep 11:53 /dev/nvidia0
crw-rw----  1 root video 0xac  9 Sep 11:53 /dev/nvidiactl

and the user in the podman container
has the same groups as on freebsd

so its in the freebsd video group
and can access the nvidia devices and other devices in the devfs.rules

/usr/local/bin/nvidia-smi is mounted from freebsd to the container so should work

NapoleonWils0n · Sep 9, 2026

cuda listed on freebsd for me

Code:

[i] Yes Master ?  ls -l /compat/linux/usr/lib64/libc*
-rwxr-xr-x  1 root wheel  2549264 11 Nov  2025 /compat/linux/usr/lib64/libc.so.6
-rwxr-xr-x  1 root wheel    54344 11 Nov  2025 /compat/linux/usr/lib64/libc_malloc_debug.so.0
lrwxr-xr-x  1 root wheel       14  3 Nov  2025 /compat/linux/usr/lib64/libcap.so.2 -> libcap.so.2.48
-rwxr-xr-x  1 root wheel    36296  3 Nov  2025 /compat/linux/usr/lib64/libcap.so.2.48
lrwxr-xr-x  1 root wheel       17  3 Nov  2025 /compat/linux/usr/lib64/libcom_err.so.2 -> libcom_err.so.2.1
-rwxr-xr-x  1 root wheel    23760  3 Nov  2025 /compat/linux/usr/lib64/libcom_err.so.2.1
lrwxr-xr-x  1 root wheel       17 25 May  2022 /compat/linux/usr/lib64/libcrypt.so.2 -> libcrypt.so.2.0.0
-rwxr-xr-x  1 root wheel   201832 25 May  2022 /compat/linux/usr/lib64/libcrypt.so.2.0.0
lrwxr-xr-x  1 root wheel       18 13 Nov  2025 /compat/linux/usr/lib64/libcrypto.so.3 -> libcrypto.so.3.5.1
-rwxr-xr-x  1 root wheel  6147456 13 Nov  2025 /compat/linux/usr/lib64/libcrypto.so.3.5.1
lrwxr-xr-x  1 root wheel       17  9 Jul 02:19 /compat/linux/usr/lib64/libcuda.so -> libcuda.so.595.84
lrwxr-xr-x  1 root wheel       17  9 Jul 02:19 /compat/linux/usr/lib64/libcuda.so.1 -> libcuda.so.595.84
-rw-r--r--  1 root wheel 91505672  9 Jul 02:19 /compat/linux/usr/lib64/libcuda.so.595.84

here are my config files

root config

GitHub - NapoleonWils0n/freebsd-root-xps: freebsd root dotfiles for dell xps 15 2019

freebsd root dotfiles for dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-root-xps development by creating an account on GitHub.

github.com

freebsd-root-xps/freebsd-root-xps.org at master · NapoleonWils0n/freebsd-root-xps

freebsd root dotfiles for dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-root-xps development by creating an account on GitHub.

github.com

dotfiles

GitHub - NapoleonWils0n/freebsd-dotfiles-xps: freebsd dotfiles dell xps 15 2019

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.

github.com

freebsd-dotfiles-xps/freebsd-dotfiles-xps.org at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.

github.com

pkg prime list of installed packages

freebsd-dotfiles-xps/freebsd-dotfiles-xps.org at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.

github.com

grandpa · Sep 9, 2026

I did sudo pkg install x11/linux-nvidia-libs and that gave this output from nvidia-smi so small step forward. Still no cuda in ollama though. Maybe rebuild base?

 bash-5.1$ nvidia-smi

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and

 running.

/grandpa

NapoleonWils0n · Sep 9, 2026

is that nvidia-smi output on the freebsd side

if you google "freebsd NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver."

there are some hits the first one for freebsd

L

Thread 'NVIDIA_SMI has failed because it couldn’t communicate with the NVIDIA driver in freebsd. How to solve it?'

Nov 24, 2024

NVIDIA_SMI has failed because it couldn’t communicate with the NVIDIA driver in freebsd. How to solve it?

i maybe wrong but i think T-Aoki on the forum works on the nvidia drivers

grandpa · Sep 9, 2026

NapoleonWils0n said:
is that nvidia-smi output on the freebsd side

if you google "freebsd NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver."

there are some hits the first one for freebsd

L

Thread 'NVIDIA_SMI has failed because it couldn’t communicate with the NVIDIA driver in freebsd. How to solve it?'

Nov 24, 2024

NVIDIA_SMI has failed because it couldn’t communicate with the NVIDIA driver in freebsd. How to solve it?

lijian

install fail

Replies: 2

Forum: Display Servers

i maybe wrong but i think T-Aoki on the forum works on the nvidia drivers

No, it's in the container.

I have rebuilt base x11 and ollama but still same.

Logs now say this:

sudo podman-compose logs -f
chmod: changing permissions of '/run/user/1001-writeable': Operation not permitted
chown: cannot read directory '/run/user/1001-writeable': Permission denied

/grandpa

grandpa · Sep 9, 2026

Now it gets interesting.

 pkg info | grep nvidia

linux-nvidia-libs-595.84       NVIDIA graphics libraries and programs (Linux)

nvidia-driver-595.99.02        NVIDIA graphics driver userland

nvidia-drm-612-kmod-595.99.02.1501000 NVIDIA DRM Kernel Module

nvidia-drm-kmod-595.99.02      NVIDIA DRM kernel module

nvidia-kmod-595.99.02.1501000  NVIDIA graphics driver kernel module

 pkg search linux-nvidia

linux-nvidia-libs-595.84       NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-304-304.137  NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-340-340.108  NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-390-390.157  NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-470-470.256.02 NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-580-580.173.02 NVIDIA graphics libraries and programs (Linux)

linux-nvidia-libs-devel-610.43.02 NVIDIA graphics libraries and programs (Linux)

 sudo podman exec -it -u ${USER} ollama /bin/bash

bash-5.1$ nvidia-smi

Failed to initialize NVML: Driver/library version mismatch

NVML library version: 595.84

/grandpa

NapoleonWils0n · Sep 9, 2026

did you rebuild the ollama image without destroying it first that might cause the issue

to try and fix it

1) start the container
2) bring the container down with the -v option to delete the volumes

Code:

doas podman-compose up -d
doas podman-compose down -v

then start the container and enter it

Code:

doas podman-compose up -d
doas podman exec -it -u ${USER} ollama /bin/bash

NapoleonWils0n · Sep 9, 2026

grandpa said:
Now it gets interesting.

pkg info | grep nvidia linux-nvidia-libs-595.84 NVIDIA graphics libraries and programs (Linux) nvidia-driver-595.99.02 NVIDIA graphics driver userland nvidia-drm-612-kmod-595.99.02.1501000 NVIDIA DRM Kernel Module nvidia-drm-kmod-595.99.02 NVIDIA DRM kernel module nvidia-kmod-595.99.02.1501000 NVIDIA graphics driver kernel module

pkg search linux-nvidia linux-nvidia-libs-595.84 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-304-304.137 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-340-340.108 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-390-390.157 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-470-470.256.02 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-580-580.173.02 NVIDIA graphics libraries and programs (Linux) linux-nvidia-libs-devel-610.43.02 NVIDIA graphics libraries and programs (Linux)

sudo podman exec -it -u ${USER} ollama /bin/bash bash-5.1$ nvidia-smi Failed to initialize NVML: Driver/library version mismatch NVML library version: 595.84

/grandpa

looks like you have a mismatch between the driver version and the libraries

you could try updating and reinstalling the drivers

maybe something like this

Code:

sudo pkg update
sudo pkg upgrade linux-nvidia-libs nvidia-driver

and rebooting

grandpa · Sep 9, 2026

Well, I figured the easiest way would be to upgrade the linux-nvidia-libs to 595.99.02 since FreeBSD Ports Search lists it.

However after doing the below I am confused.

 git pull

git status

On branch main

Your branch is up to date with 'origin/main'.

ls x11/linux-nvidia-libs*

x11/linux-nvidia-libs:

Makefile        distinfo        pkg-descr       pkg-plist



x11/linux-nvidia-libs-304:

Makefile        distinfo



x11/linux-nvidia-libs-340:

Makefile        distinfo



x11/linux-nvidia-libs-390:

Makefile        distinfo



x11/linux-nvidia-libs-470:

Makefile        distinfo



x11/linux-nvidia-libs-580:

Makefile        distinfo



x11/linux-nvidia-libs-devel:

Makefile        distinfo

What a kerfuffle! I think I need to sleep on this.

/grandpa

grandpa · Sep 9, 2026

Ah, now I see, it's the one without number in the ports tree. Ok installing.

/grandpa

general/other Cuda working on Freebsd using Podman - support thread

Attachments