bhyve Current state of bhyve Nvidia passthrough?

I also tested it — it just works!
FreeBSD 14.3 clean install with the D51892 patch.
Hardware: Supermicro X10DPU + GTX 1080 Ti, Windows 10 guest.
Thanks, Corvin.
 

Attachments

  • Snipaste_2025-08-15_12-34-05.png
    Snipaste_2025-08-15_12-34-05.png
    116.7 KB · Views: 75
Hi All, I have this working for linux guests (Ubuntu) and a windows 10 guest without changing the kvm string in the file below:
/usr/src/freebsd/sys/amd64/vmm/x86.c:

This is what is in my file:
C: 
static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";

Hardware details:
Dell R720 Server With:
CPU 1 - Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
- Virtualization Tech (VT) capable=YES enabled=YES
CPU 2 - Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
- Virtualization Tech (VT) capable=YES enabled=YES
Memory = 96GB DDR-3 Dual Rank Multi-bit ECC
GPU - Tesla P4 (10DE,1BB3,10DE,11D8) S:00,B:02,D:00,F:00 (Upgraded to the latest firmware)
- This is a compute only GPU, there are no input and output devices
Comparing the code changes with github reveals the below changes by Corvin:
# freebsd/freebsd-src branch: main
# Beckhoff/freebsd-src branch: phab/corvink/15.0/nvidia-wip
# Files changed:
# usr.sbin/bhyve/amd64/Makefile.inc
# usr.sbin/bhyve/amd64/pci_gvt-d.c
# usr.sbin/bhyve/pci_emul.c
# usr.sbin/bhyve/pci_passthru.c
# usr.sbin/bhyve/pci_passthru.h
# usr.sbin/bhyve/pci_passthru_quirks.c

# freebsd/freebsd-src branch: releng/14.2
# Beckhoff/freebsd-src branch: phab/corvink/14.2/nvidia-wip
# Files Changed:
# sys/dev/pci/pcireg.h
# usr.sbin/bhyve/pci_passthru.c

# freebsd/freebsd-src branch: releng/14.0
# Beckhoff/freebsd-src branch: phab/corvink/14.0/nvidia-wip
# Files Changed:
# sys/amd64/vmm/x86.c
# sys/dev/pci/pcireg.h
# usr.sbin/bhyve/pci_passthru.c
# usr.sbin/bhyve/usb_mouse.c

Below are the steps I followed for testing this:
# install git
pkg update
pkg install git


#### Create a diff / patch file for FreeBSD 14.2 release for passing nvidia GPU to linux guests
# 1. Clone the two repos
cd /usr/src
git clone https://github.com/Beckhoff/freebsd-src.git beckhoff


# 2. Checkout the correct branch
cd ../beckhoff
git checkout phab/corvink/14.2/nvidia-wip


# 3. change the bhyve_id string to work with linux guests
# -static const char bhyve_id[12] = "bhyve bhyve "; # windows
# +static const char bhyve_id[12] = "KVMKVMKVM\0\0\0"; # linux
ee sys/amd64/vmm/x86.c

# change:
# static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";

git add -u
git commit -m "Apply local patch to string in bhyve_id for Linux guests"

# 4. Add the FreeBSD source as remote repo
git remote add freebsd https://github.com/freebsd/freebsd-src.git
git fetch freebsd releng/14.2

# 5. Generate the diff for three affected files (Linux guests)
git diff --src-prefix=a/ --dst-prefix=b/ \
freebsd/releng/14.2..HEAD \
-- sys/amd64/vmm/x86.c \
sys/dev/pci/pcireg.h \
usr.sbin/bhyve/pci_passthru.c \
> ../freebsd-14.2-beckhoff-linux.patch

# 6. Generate the diff for two affected files (Windows guests)
git diff --src-prefix=a/ --dst-prefix=b/ \
freebsd/releng/14.2..HEAD \
-- sys/dev/pci/pcireg.h \
usr.sbin/bhyve/pci_passthru.c \
> ../freebsd-14.2-beckhoff-windows.patch

# 7. Inspect the patches
less ../freebsd-14.2-beckhoff-linux.patch
less ../freebsd-14.2-beckhoff-windows.patch

# 8. Apply the patches to a clean FreeBSD 14.2 source
cd ../freebsd
git apply ../freebsd-14.2-beckhoff-linux.patch
git commit -am "Apply patch for passthrough of Nvidia GPU to linux guest in bhyve"

# 9. Backup the original executables before building new ones
mkdir -p ~/bhyve
cp /usr/sbin/bhyve ~/bhyve/
cp /boot/modules/vmm.ko ~/bhyve/

# 10. Get the script for building and upgrading the software and kernel and execute it
fetch https://forums.freebsd.org/attachments/build-txt.21848/ -o ./build.sh
chmod 700 ./build.sh
./build.sh --without-bhf --verbose --src-dir=/usr/src/freebsd/

# Determine your PCI bus, slot and function
dmesg | grep -B4 vga

# 11. Add the below tunables to /boot/loader.conf to load on system boot
sysrc -f /boot/loader.conf vmm_load="YES"
sysrc -f /boot/loader.conf pptdevs="66/0/0"

# 12. reboot the system
shutdown -r now

#### Start Bhyve work
# 1. Create a tap interface for the network device in the VM to attach to.
ifconfig tap0 create
sysctl net.link.tap.up_on_open=1

# 2. create a bridge
ifconfig bridge2 create
ifconfig bridge2 addm em0 addm tap0
ifconfig bridge2 up

# 3. to use UEFI install the pkg bhyve-firmware
pkg install bhyve-firmware


# 4. Create a new zfs dataset to use for vm's:
zfs create zroot/vm
zfs create zroot/vm/ubuntu

# 5. create a zvolume
zfs create -V 30G zroot/vm/ubuntu/disk0

# 6. copy the file BHYVE_UEFI.fd to your working folder
cp /usr/local/share/uefi-firmware/BHYVE_UEFI.fd /zroot/vm/ubuntu/linuxguest_VARS.fd

# 7. download Ubuntu
fetch -o /zroot/vm/ubuntu/ubuntu-24.04.2-live-server-amd64.iso https://releases.ubuntu.com/24.04.2/ubuntu-24.04.2-live-server-amd64.iso?_gl=1*qc99qr*_gcl_au*NTczODEwNTIuMTc1MTc0NDc2Ny4xNTAzMzM2MDYzLjE3NTE3NDUwNDQuMTc1MTc0NTA0NA..

# 8. change to the directory
cd /zroot/vm/ubuntu

# 9. execute the bhyve command
bhyve -c 4 -m 8192M -H -A -P -S -W -w \
-s 0,hostbridge \
-s 3,ahci-cd,./ubuntu-24.04.2-live-server-amd64.iso \
-s 4,ahci-hd,/dev/zvol/zroot/vm/ubuntu/disk0 \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
ubuntu-24-04-2

# 10. Follow the steps to install ubuntu and power off the system or reboot once it prompts

# 11. recreate the bhyve session this time passing the GPU and removing the CD device
bhyve -c 4 -m 8192M -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/ubuntu/disk0 \
-s 5,virtio-net,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
ubuntu-24-04-2
# I searched using nvidia website to determine which driver to install
# turns out with ubuntu 24.04.2 the 570 series driver is recommended
# 12. Install the driver:
sudo apt install nvidia-driver-570

# 13. blacklist nouveau
sudo nano /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
sudo update-initramfs -u

# 14. Update grub for nomodeset since this is a Tesla P4
sudo nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="nomodeset"
sudo update-grub

# 15. now powerdown and restart the vm
sudo poweroff

# 16. Enter bhyve again
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/ubuntu/disk0 \
-s 5,virtio-net,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
ubuntu-24-04-2

# 17. Test nvidia-smi
nvidia-smi
# Success

# 18. Get and install plex
cd ~/
wget https://downloads.plex.tv/plex-media-server-new/1.42.1.10060-4e8b05daf/debian/plexmediaserver_1.42.1.10060-4e8b05daf_amd64.deb?_gl=1*b0riic*_gcl_au*NjQ0NjMzMTUuMTc1NTI4NDYyOQ..*_ga*MTU0MDY3MjIzNC4xNzM5NTAxNDQ1*_ga_G6FQWNSENB*czE3NTUyODQ2MjgkbzE4JGcxJHQxNzU1Mjg0NjY0JGoyNCRsMCRoMA.. -O plexmediaserver_1.42.1.deb
sudo dpkg -i plexmediaserver_1.42.1.deb

# 19. Grab some media from my share
sudo apt update
sudo apt install smbclient
sudo smbclient //192.168.3.102/data -U "AD\Clint"
timeout 300
get file.mkv
# Setup plex and verify the graphics card is available for use.
# tested selecting graphics card and transcoding everything seems to work perfect.
# manually selecting the quality causes issues. Seems like a plex issue. Automatically selecting quality works perfectly.

#### Test building FreeBSD for windows to verify ZioMario suggestion that this is required.
# 1. backup files /boot/modules/vmm.ko and /usr/sbin/bhyve
mv /boot/modules/vmm.ko ~/bhyve/vmm_lin.ko
mv /usr/sbin/bhyve ~/bhyve/byhve_lin

# 2. change the bhyve_id string for windows systems
ee /usr/src/freebsd/sys/amd64/vmm/x86.c
static const char bhyve_id[12] = "bhyve bhyve ";
git add -u
git commit -m "Apply local patch to string in bhyve_id for Windows guests"

# 3. Rebuild for windows
./build.sh --without-bhf --verbose --src-dir=/usr/src/freebsd/

# 4. reboot after building is complete
shutdown -r now

# 5. Enter ubuntu bhyve again
cd /zroot/vm/ubuntu
bhyve -c 4 -m 8192M -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/ubuntu/disk0 \
-s 5,virtio-net,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
ubuntu-24-04-2

# 6. Test nvidia-smi
nvidia-smi
# This time it fails.

# 7. Save the bhyve executable as the windows Version
cp /boot/modules/vmm.ko ~/bhyve/vmm_win.ko
cp /usr/sbin/bhyve ~/bhyve/byhve_win

# 8. Create a new zfs dataset for my windows vm:
zfs create zroot/vm/win10

# 9. create a zvolume
zfs create -V 60G -b 32K zroot/vm/win10/disk0

# 10. copy the file BHYVE_UEFI.fd to your working folder
cp /usr/local/share/uefi-firmware/BHYVE_UEFI.fd /zroot/vm/win10/linuxguest_VARS.fd

# 11. Download the media creation tool from windows and create a bootable iso.
# I drop the bootable iso on my samba share

# 12. get the windows iso using samba client may require to install smbclient
smbclient //192.168.3.102/share -U AD\\Clint
timeout 300
get Windows.iso /zroot/vm/win10/

# 13. change to the directory
cd /zroot/vm/win10

# 14. execute the bhyve command, the virtio-net doesn't work, but this is desirable as I don't want to sign in with a M$ account.
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 3,ahci-cd,./Windows.iso \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10

# 15. Install windows, after installation remove the cd drive and reboot with no internet
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,virtio-net,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10

# 16. Restart the vm with internet now using e1000 device attached to the tap0
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,e1000,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10

# 17. Download and install nvidia drivers and plex media server
# Once complete add some media from my samba share by copying a video file for testing transcoding
# map a drive with AD\Clint as the user and enter my password and check the check box to remember the login details.
# The nvidia driver recommended by nvidia website 580.88 doesn't work.
# I used the nvidia driver 538.78 and it seems to work.

# 18. Restart the vm
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,e1000,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10

# 19. Open an elevated command prompt
nvidia-smi
# success.

# 20. Setup plex and verify the graphics card is available for use.
# tested selecting graphics card and transcoding, everything seems to work perfect.
# manually selecting the quality causes issues. Seems like a plex issue. Automatically selecting quality works perfectly. Shutdown the vm.

# 21. replace bhyve and vmm.ko with the linux version and reboot
cp -i ~/bhyve/bhyve_lin /usr/sbin/bhyve
cp -i ~/bhyve/vmm_lin.ko /boot/modules/vmm.ko
cp -i ~/bhyve/vmm_lin.ko /boot/kernel/vmm.ko

# 22. Restart the VM to test with the linux built vmm.ko and bhyve.
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,e1000,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10

# 23. Setup plex and verify the graphics card is available for use.
# tested selecting graphics card and transcoding, everything seems to work perfect.
# manually selecting the quality causes issues. Seems like a plex issue. Automatically selecting quality works perfectly. Shutdown the vm.

# Just to be sure, I apply the patch again:
ee /usr/src/freebsd/sys/amd64/vmm/x86.c
static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";
cd /usr/src/freebsd/
git add -u
git commit -am "Apply patch for passthrough of Nvidia GPU to linux guest in bhyve"
./build.sh --without-bhf --verbose --src-dir=/usr/src/freebsd/
shutdown -r now

# enter windows bhyve again to see if it is working.
cd /zroot/vm/win10
bhyve -c 16 -m 32G -H -A -P -S \
-s 0,hostbridge \
-s 4,ahci-hd,/dev/zvol/zroot/vm/win10/disk0 \
-s 5,e1000,tap0 \
-s 6,passthru,66/0/0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1080,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,./linuxguest_VARS.fd \
win10
# test plex again and nvidia-smi
# works as expected.

ZioMario, I can confirm there is no need to create a different vmm.ko and bhyve executables for Nvidia GPU passthrough in both Windows and Linux VM's at least on my hardware. In the file /usr/src/freebsd/sys/amd64/vmm/x86.c Windows seems to work fine with either of the strings below:
C: 
static const char bhyve_id[12] = "bhyve bhyve ";
static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";
However, Linux only works only with the string:
C: 
static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";

I have attached the patch file that works for both Linux and Windows guests. This patch should be applied to a fresh FreeBSD 14.2 source.
cd /usr/src/
git clone https://github.com/freebsd/freebsd-src.git
git fetch freebsd releng/14.2
cd freebsd
git apply freebsd-14.2-beckhoff-linux.patch.txt

grab the build.sh script and build everything like I did in the previous steps.
 

Attachments

Last edited:
FreeBSDNewUser said:
Hi All, I have this working for linux guests (Ubuntu) and a windows 10 guest without changing the kvm string in the file below:
/usr/src/freebsd/sys/amd64/vmm/x86.c:
Click to expand...
Maybe the string you mentioned (static const char bhyve_id[12] = "bhyve bhyve ";) should be per-VM variable.
Make it to array of arrays and store all known possible pattern, and select one via per-VM config would be wanted?
C: 
static const char bhyve_id[2][12] = {
   "bhyve bhyve ",
   "KVMKVMKVM\0\0\0"
};
 
T-Aoki said:
Maybe the string you mentioned (static const char bhyve_id[12] = "bhyve bhyve ";) should be per-VM variable.
Make it to array of arrays and store all known possible pattern, and select one via per-VM config would be wanted?
C: 
static const char bhyve_id[2][12] = {
   "bhyve bhyve ",
   "KVMKVMKVM\0\0\0"
};
Click to expand...
For me there is no need. The "KVMKVMKVM\0\0\0" string works in both Windows and Linux Bhyve guests. Can anyone else confirm they need to change the string to "bhyve bhyve " for windows guest to work?
 
FreeBSDNewUser said:
Can anyone else confirm they need to change the string to "bhyve bhyve " for windows guest to work?
Click to expand...
Not sure for Windows, but Linux and Windows are NOT everything run on bhyve.
And your need to edit source code from "bhyve bhyve " to "KVMKVMKVM\0\0\0" is why the variabe need to be array of array (array of string) and need mechanism to choose one from it is mandatory and need fix.
Anything correctly detect bhyve would want "bhyve bhyve ".

On the other hand, when FreeBSD runs as a guest (not host), FreeBSD checks the host from the candidates defined in sys/x86/x86/identcpu.c.
In my humble opinion, at least anything defined in vm_cpuids[] there should be able to chosen as candidates per-VM. But implementing it would beyond me.
 
T-Aoki said:
Not sure for Windows, but Linux and Windows are NOT everything run on bhyve.
And your need to edit source code from "bhyve bhyve " to "KVMKVMKVM\0\0\0" is why the variabe need to be array of array (array of string) and need mechanism to choose one from it is mandatory and need fix.
Anything correctly detect bhyve would want "bhyve bhyve ".

On the other hand, when FreeBSD runs as a guest (not host), FreeBSD checks the host from the candidates defined in sys/x86/x86/identcpu.c.
In my humble opinion, at least anything defined in vm_cpuids[] there should be able to chosen as candidates per-VM. But implementing it would beyond me.
Click to expand...
Feel free to test on other OS's, I am just stating that the messaging I received regarding the need to change the string different from what is in Corvin's original patch for a Windows guest to work is misleading.
 
Last edited:
ClintBSD said:
Is there a way to update the vmm manpage with this information?
Click to expand...
Opening PR for it would be the way to go if there's none for it.

And note that modifying "bhyve bhyve " to "KVMKVMKVM\0\0\0" means that guests are mis-understanding "oh, I'm running on Linux KVM!", which can cause problems other than passthrough in the future. The best way is to make guest OS'es properly recognize bhyve and support it, but it's on each OS'es developers, unfortunately.
 
T-Aoki said:
Opening PR for it would be the way to go if there's none for it.

And note that modifying "bhyve bhyve " to "KVMKVMKVM\0\0\0" means that guests are mis-understanding "oh, I'm running on Linux KVM!", which can cause problems other than passthrough in the future. The best way is to make guest OS'es properly recognize bhyve and support it, but it's on each OS'es developers, unfortunately.
Click to expand...
I will open a PR for the vmm manpage.

The modification of the string from "bhyve bhyve " to "KVMKVMKVM\0\0\0" was in Corvin's branch for FreeBSD release 14.0 and was in the patch files that he provided for 14.2 early on in this post.
https://github.com/Beckhoff/freebsd-src/tree/phab/corvink/14.0/nvidia-wip/sys/amd64/vmm

If there is a branch or patch that can be used, and works for linux guests without changing that string please someone let me know so that I can test it out.
 
ClintBSD said:
If there is a branch or patch that can be used, and works for linux guests without changing that string please someone let me know so that I can test it out.
Click to expand...
As far as I know, there's none, unfortunately.

And running guests on bhyve that currently want "KVMKVMKVM\0\0\0" without patching FreeBSD host side means requiring the guest OS (Linux, Windows,...) to support bhyve officially. In this case, guest OS woult look for the string, finds "bhyve bhyve ", then understand it's bhyve. I believe it SHALL be done, but it's 100% on guest OS side.

So ugly workaround to switch the fingerprint string would be needed until ALL OS IN THE WILD OFFICIALLY SUPPORT BHYVE.

Currently, only way to do so is to rewrite the string on source code. So I'm stating that FreeBSD needs to have multiple strings and mechanism to choose one from them to expose to each guests per-VM-instance basis on comment #131, defaulting current "bhyve bhyve ".
 
ZioMario said:
I'm asking this because at the moment I'm still using two different executables (bhyve-win & bhyve-lin : depending on the operating system I want as a guest) and the same vmm.ko module (with the KVM string embedded inside),but if I want to passhtru my GPU to a Windows vm using the executable bhyve-lin (patched by Corvin),it will not work ; Windows will report Error 43,showing the same behaviour that it showed if the string hadn't been applied). Never understood why.
Click to expand...
Sorry, I have no idea what you're trying to say here. Does windows not work with the bhyve identifier as "KVMKVMKVM"?
Click to expand...
c-c-c-c and ZioMario, Windows does work with the bhyve identifier as "KVMKVMKVM\0\0\0". I tested by replacing both vmm.ko and bhyve executables.
 
ClintBSD said:
c-c-c-c and ZioMario, Windows does work with the bhyve identifier as "KVMKVMKVM\0\0\0". I tested by replacing both vmm.ko and bhyve executables.
Click to expand...

The problem that I still have is that I should use two bhyve executables. The first one allows to passthru the nvidia gpu to a Linux vm ; the second one to a Windows vm. I tried the executable that should allow to pass the gpu to both the os in one only shot,but it does not work for me in both ways. I don't remember if it allows to pass it on the linux vm or in the windows vm.
 
ZioMario said:
The problem that I still have is that I should use two bhyve executables. The first one allows to passthru the nvidia gpu to a Linux vm ; the second one to a Windows vm. I tried the executable that should allow to pass the gpu to both the os in one only shot,but it does not work for me in both ways. I don't remember if it allows to pass it on the linux vm or in the windows vm.
Click to expand...
When I replaced vmm.ko and bhyve I needed to replace with the below commands:
cp -i ~/bhyve/bhyve_lin /usr/sbin/bhyve
cp -i ~/bhyve/vmm_lin.ko /boot/modules/vmm.ko
cp -i ~/bhyve/vmm_lin.ko /boot/kernel/vmm.ko
 
ClintBSD said:
When I replaced vmm.ko and bhyve I needed to replace with the below commands:
cp -i ~/bhyve/bhyve_lin /usr/sbin/bhyve
cp -i ~/bhyve/vmm_lin.ko /boot/modules/vmm.ko
cp -i ~/bhyve/vmm_lin.ko /boot/kernel/vmm.ko
Click to expand...

It seems that you need to use two vmm files and two bhyve files,one for Linux and one for Windows ? Actually I need one vmm file and two bhyve executables. Situations seems to be different.
 
ZioMario said:
It seems that you need to use two vmm files and two bhyve files,one for Linux and one for Windows ? Actually I need one vmm file and two bhyve executables. Situations seems to be different.
Click to expand...
No, I actually only need the one that is built for linux because it also works with Windows.
 
ZioMario said:
bhyve_lin ? vmm_lin.ko ? why *._lin ?
Click to expand...
Because I followed your guidance and created one for linux and one for windows. After testing windows with the linux versions I found out I don't need the windows version. Please refer to my previous posts on this. It also seems /boot/loader.conf needs some tunables to pass thru graphics cards on some systems.

hw.vmm.amdvi.enable="1" # for amd IOMMU

hw.vmm.amdvi.enable is from the webpage vincent mentioned.
Post edited I no longer need the tunable hw.vmm.enable_vtd=1
 
Last edited:
FreeBSD 14.2-RELEASE is going EOL and I should upgrade it to 14.3. I have already done it to one of my installations. And then I copied the usual bhyve files from the 14.2 to the 14.3. The bhyve executables work,but the kernel module vmm.ko compiled for the 14.2 does not work for the 14.3. It's a crucial point because in 1 month I can't use the 14.2 anymore and the 15.0 is not yet ready and mature. And I need to have the vmn.ko for the 14.3 patched by Corvin to be able to continue to use the passthru. Point here is that Corvin does not offer vmm.ko for FreeBSD 14.3-RELEASE. What to do in this situation ? Without his patches I can pass the GPU in a Windows VM (getting bhyve from mainline) but not on a Linux VM.
 
ClintBSD said:
I have attached the patch file that works for both Linux and Windows guests. This patch should be applied to a fresh FreeBSD 14.2 source.
cd /usr/src/
git clone https://github.com/freebsd/freebsd-src.git
git fetch freebsd releng/14.2
cd freebsd
git apply freebsd-14.2-beckhoff-linux.patch.txt

grab the build.sh script and build everything like I did in the previous steps.
Click to expand...

Does it work with FreeBSD 14.3 ?
 

Hello.

I have followed these instructions on FreeBSD 14.3-RELEASE :

Code: 
# cd /usr/src/
# git clone https://github.com/freebsd/freebsd-src.git
# git fetch freebsd releng/14.3
# cd freebsd
# git apply freebsd-14.2-beckhoff-linux.patch.txt

and I started the Windows vm with these parameters :

Code: 
bhyve -S -c sockets=8,cores=1,threads=1 -m 8G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/$vmdisk0'p2'/bhyve/img/Windows/11/Windows11.img,bootindex=1 \
-s 2,ahci-hd,/dev/nda0 \
-s 3,ahci-hd,/dev/$vmdisk5 \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 13,virtio-net,tap12 \
-s 29,fbuf,tcp=0.0.0.0:5912,w=1600,h=950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:12 < /dev/null & sleep 5 && vncviewer 0:12 &

and I didnt' see errors when I patched the source code. I've been able to passthru my nvidia GPU inside a Linux VM,but not inside a Windows one. I don't see the error 43,but my monitor does not turn on. Can someone guess why ?


It works. Thanks.
 
You must log in or register to reply here.
Back
Top