Solved libc6-shim-20240512 ffmpeg nvenc working but issues with obs and handbrake

NapoleonWils0n

NapoleonWils0n

the libc6-shim package installs a program called nv-slgrun
which can be used with various applications to enable nvenc encoding and cuda

for example osb studio, ffmpeg, handbrake, blender

nv-slgrun was working with the nvidia 535 driver
which let you use nvenc encoding with obs studio and ffmpeg or with openai-whisper and davinci resolve in a jail

but nv-slgrun wasnt working with the nvidia 550 driver
and 14.1 ( which im running ) doesnt have a package for the 535 driver so you cant downgrade

the libc6-shim port was updated last month
however im running the quarterly package branch which doesnt have the latest version of the package yet

FreshPorts -- emulators/libc6-shim: ABI-compatible glibc shim library

A shim library providing limited glibc ABI compatibility on top of FreeBSD libc. It allows loading shared objects compiled on Linux and dynamically linked to glibc into native FreeBSD processes.
www.freshports.org www.freshports.org


so i used poudriere with HEAD to get the latest version

port list

Code: 
emulators/libc6-shim

building the libc6-shim port

Code: 
poudriere bulk -j freebsd_14-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list

building the port took about 10 minutes to install

i tried nvenc encoding with ffmpeg, obs studio and handbrake

1 - ffmpeg

using the nv-sglrun command and then ffmpeg with hevc_nvenc codec

Code: 
nv-sglrun ffmpeg -i input.mp4 -c:v hevc_nvenc -c:a aac output.mp4

this worked and created a new video with the hevc_nvenc codec
with no issue with the video or audio

2 - obs studio

starting obs studio with the follow code in a desktop entry

Code: 
Exec=sh -c 'LD_LIBMAP="`nv-sglrun printenv LD_LIBMAP | grep -v libGL`" obs'

i created a new profile in obs studio and ran the setup wizard
which checks your hardware and then applies those settings to obs

in the obs output settings you can either choose simple or advance options for encoding the video

i have tested both the simple and advanced options for nvenc encoding
and unfortunately neither of them worked

obs studio output mode simple

20240613_20h37m38s_grim.png


obs studio output mode advanced

20240613_20h42m02s_grim.png



when you click start recording i came across these errors

20240613_20h31m23s_grim.png



and this error

20240613_20h25m26s_grim.png


with this error in the obs studio logs

Code: 
20:38:58.158: Failed to inhibit: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.portal.Inhibit” on object at path /org/freedesktop/portal/desktop
20:38:58.556: [NVIDIA NVENC H.264 (FFmpeg) encoder: 'advanced_video_recording'] ffmpeg_video_encode: Error encoding: Invalid argument
20:38:58.556: Error encoding with encoder 'advanced_video_recording'
20:38:58.559: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/djwilcox/video/2024-06-13_20-38-57.mkv' stopped

trying a few different options i managed to get obs to work when clicking start
and you could set the timer indicating that its recording

after stopping record the video only had a file size of 48k
with was green

3 - handbrake

i also tried using handbrake with the following code in a desktop entry

Code: 
Exec=sh -c 'LD_LIBMAP="`nv-sglrun printenv LD_LIBMAP | grep -v libGL`" ghb %f'

and then using a nvenc encoder preset to convert a 30 second clip

handbrake would start the encode and then get stuck
and you would have to kill handbrake

and also had this error

Code: 
(fr.handbrake.ghb:40457): Gtk-WARNING **: 20:19:56.142: Calling org.freedesktop.portal.Inhibit.Inhibit failed: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.portal.Inhibit” on object at path /org/freedesktop/portal/desktop
[20:19:56] gtkgui: Preset: /Hardware/H.265 NVENC 1080p
[20:19:56] Starting work at: Thu Jun 13 20:19:56 2024

obs and handbrake seem to have a similar error

Code: 
org.freedesktop.portal.Inhibit

not sure if thats an issue

so to recap libc6-shim-20240512 works with ffmpeg
but in my testing i couldnt enable nvenc encoding with obs studio or handbrake

i tried using both scpy and dmabuf to capture the display in obs studio
using both the simple and advanced nvenc encoder settings

both obs studio and handbrake use ffmpeg which is working but they arent
which is odd

nv-sglrun nvidia-smi works

Code: 
Yes Master ? nv-sglrun nvidia-smi
shim init
Thu Jun 13 22:13:28 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+

github.com

GitHub - shkhln/libc6-shim: Cheap glibc knockoff

Cheap glibc knockoff. Contribute to shkhln/libc6-shim development by creating an account on GitHub.
github.com github.com
 
Eureka

I found a way to enable nvenc hevc encoding on obs with the 550 nvidia driver
using libc6-shim-20240512 built with poudriere ( because the latest version isnt in quarterly )

ffmpeg hevc_nvenc is working on the command with the 550 driver

Code: 
nv-sglrun ffmpeg -i input.mp4 -c:v hevc_nvenc -c:a aac output.mp4

so thats the codec we need to use with obs
but the trick is actually setting the encoder in obs studio

in the ouput section of the settings you have to set the output mode to advanced
and the type to custom ffmpeg command

the crucial step is enabling the "Show all codecs (even if potentially incompatible)" checkbox
this allows you to select the hevc_nvenc - NVIDIA NVENC hevc encoder

the nvenc h264 codec didnt work for me only the nvenc hevc codec

i have been through all the permutations of settings
and this is the only way to enable nvenc hevc encoding

the steps to enable nvenc hevc encoding on obs

1 - open the obs settings
2 - go to the output section
3 - output mode to advanced
4 - type to custom output (ffmpeg)
5 - check "Show all codecs (even if potentially incompatible)" # this is the secret sauce
6 - video encoder set to hevc_nvenc - NVIDIA NVENC hevc encoder
7 - audioencoder set to aac


20240614_14h45m24s_grim.png
 
Handbrake settings now working

the default preset for nvnenc encoding in handbrake uses AV1 (SVT) video codec

but using that preset didnt work and handbrake was processing 2 frames a second
and then just hang and never finish the encode

AV1 (SVT) video codec

20240614_15h38m31s_grim.png



so i duplicated the Hardware/H265 NVENC 1080p preset
and in the video tab changed the video encoder from AV1 (SVT) video codec

to H.265 (x265) which works

20240614_15h37m41s_grim.png
 
ffmpeg command to use cuda and hevc nvenc

Code: 
nv-sglrun ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda \
-i input.mp4 -c:a copy -c:v hevc_nvenc -b:v 5M output.mp4

taken from the nvidia documentation and tweaked a bit
i change the video encoder from h264_nvenc to hevc_nvenc

h264_nvenc doesnt work but hevc_nvenc does
i also prefixed the command with nv-sglrun

Using FFmpeg with NVIDIA GPU Hardware Acceleration

Archive version of Video Codec SDK
docs.nvidia.com docs.nvidia.com
 
You must log in or register to reply here.
Back
Top