Solved Trying to understand why only some kind of vmm.ko module allows to passthru my nvidia GPU to a WIndows 11 vm

ziomario said:
Please tell me the first commit of the FreeBSD 14.0-RELEASE p6.
Click to expand...
Based on this text from the Supported Releases section of FreeBSD Security Information:
The Security Branch tags have names like releng/13.2. The corresponding builds have names like FreeBSD 13.2-RELEASE-p1.
Click to expand...
and looking through releng/14.0 commits I'm guessing it's d338712beb16ad7740bbd00bd93299a131a68045
It might be best to wait for @cracauer to confirm though.

I'm also suffering from Window 10 VM not working since upgrading from Corvin's version of FreeBSD-13.2 to 14.1-RELEASE.
In my case, I'm trying to pass through the iGPU of the i7-9700T.

Could you post which version of edk2-bhyve you have installed?
For me, it is:
Bash: 
# pkg info -I edk2-bhyve
edk2-bhyve-g202308_4           EDK2 Firmware for bhyve
#

As you have an i9-9900K I'm secretly hoping you've stumbled upon a bug common to us both!
 
I want the double confirmation that the GPU can be passed on 14-RELEASE-p6 using its own vmm.ko module...it is also true that if on the p7 I use the vmm.ko that works for p6,it works even on p7. I suspect that it will work even on p8. For sure it will stop working starting from 14.1,unless a module that works for 14.0 can be adapted for 14.1 in some way. My theory is that everytime that I need to recompile that file,it will not work. Maybe has been compiled in any special way ?

Can you confirm the two exact revisions that worked and didn't work?
Click to expand...

I don't understand what you need to know. At the moment it seems that it works only on 14.0-RELEASE-p6.
 
redmog said:
Based on this text from the Supported Releases section of FreeBSD Security Information:

and looking through releng/14.0 commits I'm guessing it's d338712beb16ad7740bbd00bd93299a131a68045
It might be best to wait for @cracauer to confirm though.

I'm also suffering from Window 10 VM not working since upgrading from Corvin's version of FreeBSD-13.2 to 14.1-RELEASE.
In my case, I'm trying to pass through the iGPU of the i7-9700T.

Could you post which version of edk2-bhyve you have installed?
For me, it is:
Bash: 
# pkg info -I edk2-bhyve
edk2-bhyve-g202308_4           EDK2 Firmware for bhyve
#

As you have an i9-9900K I'm secretly hoping you've stumbled upon a bug common to us both!
Click to expand...

You should ask to bileslav ; he has been able to passthru his GPU to Windows 10 and to Linux at the same time,using one only vmm.ko module and one only bhyve executable. I haven't been able to do it. I'm using one only vmm.ko module,but two different bhyve executable. The problem for me is that the Corvin's patches interferes and they don't allow to passthru my GPU to a Windows vm. So,the bhyve executable that I'm using , when I want to virtualize Windows,is not patched with the Corvin's patches.

---> As you have an i9-9900K I'm secretly hoping you've stumbled upon a bug common to us both!

Which bug in common we have ?
 
redmog said:
Based on this text from the Supported Releases section of FreeBSD Security Information:

and looking through releng/14.0 commits I'm guessing it's d338712beb16ad7740bbd00bd93299a131a68045
It might be best to wait for @cracauer to confirm though.

I'm also suffering from Window 10 VM not working since upgrading from Corvin's version of FreeBSD-13.2 to 14.1-RELEASE.
In my case, I'm trying to pass through the iGPU of the i7-9700T.

Could you post which version of edk2-bhyve you have installed?
For me, it is:
Bash: 
# pkg info -I edk2-bhyve
edk2-bhyve-g202308_4           EDK2 Firmware for bhyve
#

As you have an i9-9900K I'm secretly hoping you've stumbled upon a bug common to us both!
Click to expand...

d338712beb16ad7740bbd00bd93299a131a68045 =14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #15 n265417-d338712beb16 = NO.

This is the version that works :

FreeBSD blackbox.localdomain 14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #0

so,I need the commit of that version. redmog : you gave me #15
 
Pretty sure the #15 represents your local build number. See this thread about it:
forums.freebsd.org

Kernel build number

Hello, What does the #0, #1, etc. found in the output of uname -v mean, exactly? I guess it's a kernel build number? Official builds all seem to be #0. I currently have FreeBSD 13.1-STABLE #4 stable/13-n250651-85983255c15. I upgraded my system today and went from a FreeBSD 13.1-STABLE #2 to a...
forums.freebsd.org forums.freebsd.org

---> As you have an i9-9900K I'm secretly hoping you've stumbled upon a bug common to us both!

Which bug in common we have ?
Click to expand...
I meant that, unlike bileslav who is using a Ryzen 5900X, we are both using Intel 9th Gen CPUs and so maybe that is a common aspect in a potential bug that you might find.

A while back you posted:
14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #32 releng/14.0-n265396-06497fbd52e2-dirty
Is this from your working environment?

I ask because the commit hash starting 06497fbd52e2 seems very much like:
14.0-RELEASE-p2 FreeBSD 14.0-RELEASE-p2 #0 releng/14.0-n265396-06497fbd52e2

So you might also want to try commit 06497fbd52e2f138b7d590c8499d9cebad182850 just in case?
 
ziomario we need to use precise language here.

The p<n> numbers are not precise since they are not tagged in the git tree.

Could you please state which commit hashes you tested and the result?
 
redmog said:
Pretty sure the #15 represents your local build number. See this thread about it:
forums.freebsd.org

Kernel build number

Hello, What does the #0, #1, etc. found in the output of uname -v mean, exactly? I guess it's a kernel build number? Official builds all seem to be #0. I currently have FreeBSD 13.1-STABLE #4 stable/13-n250651-85983255c15. I upgraded my system today and went from a FreeBSD 13.1-STABLE #2 to a...
forums.freebsd.org forums.freebsd.org

I meant that, unlike bileslav who is using a Ryzen 5900X, we are both using Intel 9th Gen CPUs and so maybe that is a common aspect in a potential bug that you might find.
Click to expand...

Why are you talking about the cpu if we are trying to passthru correctly our gpu ? I don't know how much they are correlated.

---> So you might also want to try commit 06497fbd52e2f138b7d590c8499d9cebad182850 just in case ?

Good idea. I'm trying it now.
 
cracauer@ said:
ziomario we need to use precise language here.

The p<n> numbers are not precise since they are not tagged in the git tree.

Could you please state which commit hashes you tested and the result?
Click to expand...

14-RELEASE #0 = f9716eee8ab45ad906d9b5c5233ca20c10226ca7 = NO
14-RELEASE-p7 = 70eb00f17b310f599b60939c1afa326c7b2c390c = NO
14-RELEASE-p6 #15 = d338712beb16ad7740bbd00bd93299a131a68045 = NO
14-RELEASE-p6 #32 = 06497fbd52e2f138b7d590c8499d9cebad182850 =

Code: 
# git checkout 06497fbd52e2f138b7d590c8499d9cebad182850
# make -j4 buildworld buildkernel

work in progress...

commit 025a5f6b2540682c69ac63f7754ce74b43441034 : kernel panic
commit a0f02252c4175b5336c215cc7774c3ea08873476 : NO
commit 9e48b627aed346bf5e950134a581218d3097eb7c : NO
 
The point is that the vmm.ko file has been given to me by bileslav. He told me the version of FreeBSD that he was using and it was the same as mine,14.0-p6. Infact my GPU started working as soon as I have exchanged mine with his. Now I'm trying to find,on the FreeBSD servers,the version of vmm.ko + the relative FreeBSD version that uses it,because I want to check if the same /and official/ version works or not or if the only file which make the GPU correctly work is the file that bileslav gave to me. Have you caught the point ? Until now I didn't find any vmm.ko offered by the various FreeBSD versions I tried working out of the box. But if I exchange the vmm.ko that's offered by them with mine,I can passthru my GPU,even of p7. I suppose it works like that until the end of the 14.0 branch. The bileslav file always works,the default vmm.ko files offered by FreeBSD,never. But if bileslav told that it comes from 14.0-p6,should I believe to him or not ? Something is not linear here. I also add that I'm pretty sure that the "magic" is inside the vmm.ko file of bileslav,because it starts working only when I use it. I mean,I tried to use various versions of the bhyve* files located under /usr/sbin,but they don't make any difference. The GPU passthru happens only if I use the bileslav vmm.ko damn file. Is it useful for you to know his sha256 to gather more informations about it ?

Code: 
# sha256sum vmm.ko
c57645e8d1a43714bb899813567f4678dddd73ca55e5745f77daea8241126d48  vmm.ko
 
ziomario said:
14-RELEASE-p6 #15 = d338712beb16ad7740bbd00bd93299a131a68045 = NO
Click to expand...
ziomario said:
The bileslav file always works,the default vmm.ko files offered by FreeBSD,never. But if bileslav told that it comes from 14.0-p6,
Click to expand...
How did he update to p6? From source, freebsd-update?



Just to make sure, did you make cleanworld after every commit checkout, before make buildworld buildkernel?

build(7)
Code: 
     cleanworld       Attempt to clean up targets built by a preceding
                      buildworld, or similar step, built from this source
                      directory.
What is build from buildworld|kernel is written in the object directory /usr/obj. If should be empty before building after every commit checkout.



You are concentrating at the moment on 14.0 and 14.1, maybe I missed it, but have you or bileslav ever tried on a 15.0-CURRENT system?
 
ziomario said:
The point is that the vmm.ko file has been given to me by bileslav. He told me the version of FreeBSD that he was using and it was the same as mine,14.0-p6. Infact my GPU started working as soon as I have exchanged mine with his. Now I'm trying to find,on the FreeBSD servers,the version of vmm.ko + the relative FreeBSD version that uses it,because I want to check if the same /and official/ version works or not or if the only file which make the GPU correctly work is the file that bileslav gave to me. Have you caught the point ? Until now I didn't find any vmm.ko offered by the various FreeBSD versions I tried working out of the box. But if I exchange the vmm.ko that's offered by them with mine,I can passthru my GPU,even of p7. I suppose it works like that until the end of the 14.0 branch. The bileslav file always works,the default vmm.ko files offered by FreeBSD,never. But if bileslav told that it comes from 14.0-p6,should I believe to him or not ? Something is not linear here. I also add that I'm pretty sure that the "magic" is inside the vmm.ko file of bileslav,because it starts working only when I use it. I mean,I tried to use various versions of the bhyve* files located under /usr/sbin,but they don't make any difference. The GPU passthru happens only if I use the bileslav vmm.ko damn file. Is it useful for you to know his sha256 to gather more informations about it ?

Code: 
# sha256sum vmm.ko
c57645e8d1a43714bb899813567f4678dddd73ca55e5745f77daea8241126d48  vmm.ko
Click to expand...
By now, I'm a bit tired of this nonsense, bro. Really.

1.png

2.png

3.png

4.png
 
bileslav said:
freebsd-update


Nope
Click to expand...

bileslav : I want to clarify that I never thought that you did something of malicious with the files that you gave to me. I suspect that,in some way,the vmm.ko file that you gave to me cannot be found in the FreeBSD servers and it can't be replicated because there are some differences between your and my machine,for example...or from your system and mine,even if we ran the same FreeBSD version. Maybe this is totally wrong,but I think its legit to think it unless someone more experienced than me tell that it can't be. Anyway,my researches aren't finished yet and maybe I will find it.
 
T-Daemon said:
Right.

Building from source for some time now, last time I checked I had to cleanworld. To reduce build time I have meta mode enabled, but forgot about it.
Click to expand...

I never used make cleanworld. I don't think it makes some difference because everytime I reboot the machine without issuing that command,I see that the FreeBSD version is changed,the old vmm.ko is erased and a new vmm.ko takes its place and even the bhyve files under /usr/sbin are replaced.
 
redmog said:
So you might also want to try commit 06497fbd52e2f138b7d590c8499d9cebad182850 just in case?
Click to expand...

I did it :

Code: 
git checkout 06497fbd52e2f138b7d590c8499d9cebad182850
make -j8 buildworld buildkernel
make -j8 installworld installkernel

but this is the FreeBSD version that I've got :

Code: 
FreeBSD marietto 14.0-RELEASE-p2 FreeBSD 14.0-RELEASE-p2 #16 n265396-06497fbd52e2:
Wed Jul 10 14:20:44 CEST 2024     marietto@marietto:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

I expected to have 14.0-RELEASE-p6. I also see that "dirty" is missing. This is what I wanted to achieve :

Code: 
14.0-RELEASE-p6 FreeBSD 14.0-RELEASE-p6 #32 releng/14.0-n265396-06497fbd52e2-dirty

but :

Code: 
git checkout 06497fbd52e2f138b7d590c8499d9cebad182850-dirty

does not work. Anyway,the version of FreeBSD used by Bileslav wasn't marked as dirty. So,it makes no sense to further look in the "dirty" area.

Can I know if some investigation can be done within the vmm.ko module itself,to look for some specific memory address for example ? or something that can give more and useful informations about the reasons why it works ?

I don't have any more ideas about what to do at the moment. That file seems to be unique and it can't also be used on 14.1.

Another oddity is that Bileslav also gave me the vmm.ko that he uses on 14.1. I was hoping that it had worked like the one for 14.0, but it didn't.
 
You must log in or register to reply here.
Back
Top