bhyve Is there a method to understand inside which FreeBSD branch does my vmm.ko file come from ?

ziomario

ziomario

Hello.

After a lot of years of experiments,comparisons,installations and reinstallations,I have been able to passhtru my GeForce RTX 2080 ti to a bhyve virtual machine running with Windows,overcoming/fixing the errors that I've got for years,errors 43 and 12,in a stable and permanent way.

The key was to use the right version of the bhyve executables and of the libraries.

Now,I would like to know how many users have been able to pass their nvidia gpu to a bhyve virtual machine running Windows. This post is not for the users who have an AMD gpu.

I created this post to realize how many users can do that and how many can't and above all,if there is a method to understand inside which FreeBSD branch does my vmm.ko file come from. Just because it is a key file and I've got it from a Reddit user. He told me that he has got it from the latest packages of FreeBSD 14,so I have installed the same version of FreeBSD and I'm using the latest packages,but my (old) vmm.ko file never worked. His version works like a charme.

I really want to know WHERE can I get the same file permanently.

Now,let's give a look at the image below :

Screenshot_2024-06-06_14-05-24.png


the working vmm.ko is called vmm-s.ko" and it is dated 2024 and a reddit user gave it to me. The 2023 version is the file put there by FreeBSD 14.0 p6 with latest packages and it does not allow the nvidia gpu to be passed trhru correctly in a Bhyve / Windows vm.

So now the question is,where I can get the vmm.ko version 2024 ?
 
Last edited:
Please share output from the command, it will help to answer your question.

Postscript: I added formatting above, to make clearer that it's a command. (Sorry, I fell out of the habit of formatting, it's annoyingly bugged.)
 
Code: 
this is the only version of vmm.ko that works ok [at least 4 me] is :

==> file /boot/kernel/vmm-win.ko
/boot/kernel/vmm-win.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), BuildID[sha1]=e99da5bd8f91eb60cbc2136751ccc0e9f0e7721e, not stripped
 
uname -v will show you the branch name and git commit hash from which /boot/kernel/kernel was built (assuming that is what you are running). vmm.ko gets built at the same time and from the same branch. Not sure what the BuildID[sha1] that file reveals is.
 
Nope. The original vmm.ko of my FreeBSD installation (14.0 RELEASE p6) wasn't capable to passthru my GPU to the Windows VM. But a reddit user gave to me another vmm.ko and it started working.
 
Why not ask the author of said file? If there is a difference, he changed & compiled it right? It sounds easy...
Or you could just check all symbols with nm -a | cut ... | sort -u and compare it with your installation, although I'm not sure that will reveal much more than maybe a backdoor here or there.
 
Alain De Vos said:
Contact the reddit user ?
Click to expand...

yes I'm in contact with him. He said that he used the same branch than me : 14.0-RELEASE p6. At this point I've performed a fresh installation of 14.0 and I've upgraded to p6 and I've checked if the bhyve executable placed automatically by FreeBSD on /boot/kernel is the same that he has and the hash code. And nope. It's not the same and it does not work for me. But the file that he gave to me works perfectly. I don't know if it is relevant or not,but he has two GPUs on his PC : AMD and nVidia. In addition the hash code of the vmm.ko that works for him is not the same than mine. But has been him to give me that file. The only "thing" that I did has been to rename it from vmm.ko to vmm-win.ko. To check the hash code does not seem a valid method to understand what's the origin of a file. I imagine that if the code is recompiled,the hash code changes. Is this correct ? Anyway inside that file there is the patch that makes the magic. I imagine that to understand why it works we should make the reverse engineering of the file.
 
Last edited:
Sorry. Sometimes I make confusion between RELEASE and CURRENT. This is my system :

Code: 
FreeBSD marietto 14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #32 releng/14.0-n265396-06497fbd52e2-dirty: Sun Jun  2 23:28:12 CEST 2024    marietto@marietto:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 1400097 1400097
 
Code: 
# freebsd-version -kru

14.0-RELEASE-p6
14.0-RELEASE-p6
14.0-RELEASE-p6

# bectl list -c creation

BE                                Active Mountpoint Space Created
default                           NR     /          227G  2023-12-04 16:08
14.0-RELEASE-p5_2024-04-01_211643 -      -          9.33G 2024-04-01 21:16
14.0-RELEASE-p6_2024-06-04_124102 -      -          53.2G 2024-06-04 12:41
 
He gave me all the bhyve* files that are inside the directory /usr/sbin,his vmm.ko file and the BHYVE* files that are inside /usr/local/share/edk2-bhyve,but I already had all these files. I mean,they were identical to mine,except for vmm.ko. His vmm.ko file was different that mine. My vmm.ko does not allow the GPU passthru,giving error 12 or 43,his file is patched and does not produce errors. I have a suspect. Since he also has one AMD gpu,maybe his vmm.ko file is tailored and patched for that GPU and it works also for the Nvidia Gpu ? maybe ?
 
I've isolated all the files that should NOT be changed,otherwise it will not work anymore. These files are the following :

Code: 
[root@marietto /boot/kernel]==> sha1sum vmm-win.ko
acb0fe9992046abea9301f050a5c1189890c3ea6 vmm-win.ko

[root@marietto /boot/kernel]==> sha1sum kernel     
1ab36c7902958f0ee2026854498a90cb171f5d64  kernel

[root@marietto /usr/sbin]==> sha1sum bhyve-win
726b2c99e3968042ff5ce904b03b80dee3019ff4  bhyve-win

[root@marietto /usr/sbin]==> sha1sum bhyvectl
33eb84464492b0a75149067931747f4c4e10aeb7  bhyvectl

[root@marietto /usr/sbin]==> sha1sum bhyveload
7e2d1dfaa0351b36434a30c174ce3d6df31971fc  bhyveload

This is the FreeBSD version that I'm using :

[root@marietto /usr/sbin]==> uname -a

FreeBSD marietto 14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #0: Tue Mar 26 20:26:20 UTC 2024     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
Code:

Today I've upgraded the packages and the passthru stopped working. Maybe because bhyve and some other components have been changed and the kernel has been recompiled.

It means that I don't know how to upgrade the system without breaking the passthru of my gpu inside a Windows 11 vm.
Can someone give me some suggestions ? Can I keep the bhyve system files as they are but at the same time can I upgrade the rest of the system ?
 
You must log in or register to reply here.
Back
Top