Well I am surprised. I was able to get full hardware accelerated transcoding for HEVC/(H.265) and H.264 working on FreeBSD 13.1 using MESA drivers and VA-API for AMD Radeon RX 580 GPU. (Although, I'm not using AMD's proprietary AMF encoder since I haven't tested it yet, the VA-API works better in my experience and have HEVC capabilities for older AMD GPUs)
I used FFMPEG from FreeBSD's ports and compiled it to do the transcoding.
Need to make sure that amdgpu drivers are installed properly and working.
I haven't seen any tutorials or implementation or explanation on the on the web to do this, so I'll leave a quick brief details on how I got it to work:
Would also like to mention the following pkgs below needs to be installed to get it to work (some might not needed):
Install vainfo:
pkg install libva-utils
Install Mesa Gallium "VAAPI" drivers, which is a Gallium driver for video decoding on AMD hardware:
pkg install mesa-gallium-va
Install Mesa Gallium "VDPAU" drivers, which is a Gallium driver for video decoding on AMD hardware
pkg install mesa-gallium-vdpau
Install OpenGL hardware acceleration drivers for DRI2+:
pkg install mesa-dri
Install Off-Screen Mesa implementation of the OpenGL API:
pkg install libosmesa
Install OpenGL libraries that support GLX and EGL clients:
pkg install mesa-libs
I did a reboot and then ran the following FFMPEG command for testing:
Capture USB stream from a V4L2 camera device (e.g. a UVC webcam), decode it and encode it as HEVC:
ffmpeg -f v4l2 -video_size 1920x1080 \
-hwaccel vaapi \
-hwaccel_device /dev/dri/renderD128 \
-hwaccel_output_format vaapi \
-i /dev/video0 -vf 'scale_vaapi=format=nv12' \
-c:v hevc_vaapi output.mp4 -y
The command worked and produced the video file and plays correctly.
This is great, I can now use FreeBSD 100% to implement a DVR/NVR system without needing to use Linux.
If this works for FFMPEG, then this means HEVC GPU transcoding should also work for OBS and many applications that uses VA-API.