general/other Cuda now working on Freebsd with a Rocky Linux Podman container

NapoleonWils0n

NapoleonWils0n

I now have Cuda working on Freebsd using a Rocky Linux Podman container

This is the python command run inside the Podman container

python3 -c "import torch; print('CUDA Available:', torch.cuda.is_available()); print('Device:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'None')

And here's the output

CUDA Available: True
Device: NVIDIA GeForce GTX 1650

Output of nvidia-smi in the Podman container

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.58.03 Driver Version: 595.58.03 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+


I just installed Whisperx which uses Cuda in a Rocky Linux Podman container for the audio transcription

Heres the crazy part there is no Nvidia driver installed in the Podman container

i map the Linuxulator libraries from Freebsd into the container,
use devfs.rules and set up matching groups and ids so the container is running with exact same user and permissions as on the Freebsd host

as well as some other tricks to get it all working

And persistent storage as well, so files are stored on the Freebsd host and arent wiped when you restart the container

Im using Rocky Linux 9.3 which is the same version as in the Linuxulator so we have maximum compatibility


So far i have in the containers i have the following working

1) ffmpeg nvenv encoding

2) Firefox with widevine for drm playback with hardware accelerated video
wayland and pulseaudio sockets mounted from Freebsd into the container to create the window and audio with zero latency
downloads directory mounted from Freebsd inside the container

3) Whisperx using Cuda for audio transcription with speaker diarization


So the upshot of this is we now have Cuda on Freebsd using Podman

I can install any command line or gui application in a Rocky Linux Podman container
Including python applications that use torch


You can see the thread here which is about getting Firefox working widevine for drm playback in a Podman container

NapoleonWils0n

general/other Thread 'Freebsd Firefox Widevine for DRM playback in Rocky Linux Podman container'


Freebsd Firefox Widevine for drm playback now working in a Rocky Linux Podman container

No Nvidia driver installed in the Podman container
works by mounting the Linuxulator directories for the Nvidia drivers from Freebsd to the Podman container

Hardware accelerated video playback

persistent Firefox settings stored on the Freebsd host so all your extensions,
bookmarks and settings are retained when you stop and restart the container

Wayland and Pulseaudio sockets for zero latency

The Podman container runs as the same user and id as on the Freebsd host
Downloads...
  • Like
  • Thanks


Im created a github repo for the podman containers

So im writing the theme tune and singing the theme tune
Little Britain reference

Screenshot of the subtitles created with Whisperx


input-[00:01:37.000]v1.jpg
 
I only have a GTX 1650 Nvidia card with 4gb of vram

But have managed to run the medium whisper model

Code: 
whisperx input.wav \
  --device cuda \
  --model medium \
  --compute_type int8 \
  --batch_size 4 \
  --threads 1 \
  --diarize \
  --highlight_words True \
  --language en

which is more accurate than the small model

[SPEAKER_01]: Mr. Macbeth is a naughty man.
[SPEAKER_01]: Do, do, do, do, do.
[SPEAKER_01]: He gone and killed another man.
[SPEAKER_01]: Do, do, do, do.
[SPEAKER_01]: I hath a good idea.
[SPEAKER_01]: Just thou keep me near, I'll be so good for the Scottish play.
 
cracauer@ said:
Can you elaborate on that?
Click to expand...

I use an env file to map the username and userid from the Freebsd host to the Podman container
So the container runs as the same user as on Freebsd and is in the same groups

i also set the HOST_DBUS_SESSION_BUS_ADDRESS from Freebsd as an env
which provides access to dbus so things like desktop notifications work in firefox running in a podman container

devfs.rules to access the gpu

Podman containers are actually oci jails which use devfs_jail=4

Code: 
[localrules=5]
add path 'da*' mode 0660 group operator
add path 'dri/*' mode 0660 group video
add path 'drm/*' mode 0660 group video
add path 'input/*' mode 0660 group video
add path 'input/event*' mode 0660 group video
add path 'nvidia*' mode 0660 group video

[devfsrules_jail=4]
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'dri*' unhide
add path 'drm*' unhide
add path 'nvidia*' unhide
add path 'speaker*' unhide

Freebsd use group 44 for the video but the container use group 39

so by adding the podman user to both video groups
and using devfs.rules which allow access to the video group

the podman users has access to the gpu

the xdg runtime dir which contains the wayland socket mounted in the container as read only
and then i create another directory which i set as the xdg runtime dir in the container

and symlink the wayland socket, pulse audio and dbus directories into that directory

pulseaudio on freebsd using default.pa creates a pulseaudio.socket in /tmp which then mounted into the container
and the container has a client.conf pulseaudio config that set the pulseaudio server to the /tmp/pulseaudio.socket

i set the Linuxulator rocky linux libraies as a volume in the podman container
/compat/linux/usr/lib64 which is then mounted to /usr/lib64/nvidia-host

then i append the that location to the LD_LIBRARY_PATH in the container

export LD_LIBRARY_PATH=/usr/lib64:/usr/lib64/nvidia-host

that way we have the original library path in the container
and the mounted Linuxulator library path that contains the nvidia libraries

I also use the dummy-uvm.so in the container

by using the devfs rules and unhiding the nvidia devices means the podman container can see the hardware
then because we are mounting the Linuxulator nvidia directory from freebsd to the container it can then see the nvidia libraries

i also mount a directory from the freebsd host like the download directory into the container
and because the podman user shares the same username, id and group id all the permissions are correct

so with firefox running in the podman container you have access to the mapped download directory from freebsd
and because the pemissions are the same when you download files using firefox in the container

they are then owned by your user on the freebsd side

thats a brief overview and there is some more withcraft to get it all working

i have to write up the some of the install documentation
but i have a full guide on setting up podman

then i can push the repo to github
and do a long techy video explaining how it all works

i have only been back on freebsd for about 2 and half weeks
and working on this for about 5 days

i have managed to get the following working

1) ffmpeg hardware acclerated encoding using nvenc

2) firefox with widevine drm and hardware acclerated playback

3) Cuda working with python and torch with whisperx

writing all the documentation is the boring bit

but just like audio is half the picture

documentation is half the project
 
You must log in or register to reply here.
Back
Top