Solved Trying to understand why only some kind of vmm.ko module allows to passthru my nvidia GPU to a WIndows 11 vm

One of the register used in the 14.1 function we are binary patching is different ( r14 vs r8). Last night I just patched it without thinking about it.

Now this module gets it right:
Code: 
Hypervisor: Origin = "KVMKVMKVM"
Now again, this is just to give you an idea what was changed in the module you provided. You could easily just change the bhyve_id in the module and compile that to achieve the same thing. And as I mentioned above it would be lucky cheap hack if that even works.

So again, this is not a solution to anything. You can try if you can fool the pass-through with this.
 

Attachments

  • vmm-new.ko.txt
    500.7 KB · Views: 17
Crivens said:
Seems to be the thread to lock over the weekend so no one runs off the rails. Is this the case? You tell me.
Click to expand...

I see that some useful progress have been done before to reach the weekend. Yeah,there is some noise,but I see more productive contributions than noises.
 
Let's assume that I want to upgrade freebsd 14.0 to 14.1 using "freebsd-update -r 14.1-RELEASE" and also let's assume that I want to modify the file x86.c swapping this line :

Code: 
static const char bhyve_id[12] = "bhyve bhyve ";

with this line :

Code: 
static const char bhyve_id[12] = "KVMKVMKVM\0\0\0";

where can I find the file x86.c ?

I mean. I know how to get the source code of freebsd 14.1. It can be done like this :

git checkout releng/14.1

but that command is good if I want to install FreeBSD 14.1 from scratch,right ? In this case I want to upgrade 14.0 to 14.1 using the script freebsd-update -r 14.1-RELEASE,but at the same time,I want to modify the file x86.c. In other words,where that script gets and where it stores the file x86.c ?
 
I haven't explained well my point. I don't want to install FreeBSD 14.1 from scratch. I want to upgrade 14.0 to 14.1,because its a faster method and because in my 14.0 I have everything ready. What should I do ? Is it enough to do :

Code: 
# git clone https://git.FreeBSD.org/src.git /usr/src
# cd /usr/src
# git checkout releng/14.1

and then...freebsd-update -r 14.1-RELEASE ?

Is this procedure correct ? (I assume that the freebsd-update script gets the source code on /usr/src,but I'm not sure...I need to know where it gets the source code and where it stores,because I need to edit the x86.c file before the upgrading starts).
 
I thought the point was you have two modules, one that works and one that doesn't. Binary diff showed us that they differ in bhyve_id.

What version of FreeBSD did you have when it was working?
What version of FreeBSD did you have when it was not working ?

Modules you shared, working and non-working one, what version of FreeBSD were they from?
 
ziomario said:
Code: 
# git clone https://git.FreeBSD.org/src.git /usr/src
# cd /usr/src
# git checkout releng/14.1

and then...freebsd-update -r 14.1-RELEASE ?

Is this procedure correct ? (I assume that the freebsd-update script gets the source code on /usr/src,but I'm not sure...I need to know where it gets the source code and where it stores,because I need to edit the x86.c file before the upgrading starts).
Click to expand...

No, that is not correct. You are mixing update-from-source and binary updates.
 
I want to compile a working module for FreeBSD 14.1. I have the module working on 14.0 (but that module has an unknown origin). I need to understand from what branch it comes from and why it works because I want to recompile it for 14.1 since I want to upgrade 14.0 to 14.1.

What version of FreeBSD did you have when it was working? : 14.0
What version of FreeBSD did you have when it was not working ? 14.1

----> Modules you shared, working and non-working one, what version of FreeBSD were they from?

vmm.ko-files - Google Drive

drive.google.com drive.google.com
 
So you do have 14.1 on your system already as you tested it and figured out it's not working. Last night you tested my module and said it's not working. Today I shared the module with "correct" bhyve_id and wanted you to test that. The module I shared (vmm-new.ko) is one for 14.1 and has the hbyve_id set to what the module of the unknown origin was using.

I wanted to you to test that to see if that actually works. Why? Well, I am curious myself if that trick really did it. Also core of this thread is about that.
If not most likely there is something else changed as the module is the same otherwise.
 
ziomario said:

https://drive.google.com/drive/folders/159I8yFyV8_gf_6HeSyUvMQY_RGGYq4SJ?usp=drive_link
Click to expand...

For the file that you put in the Working folder there:

Code: 
% file vmm.ko
vmm.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), BuildID[sha1]=e99da5bd8f91eb60cbc2136751ccc0e9f0e7721e, not stripped
%
 
ziomario would you like to try a copy of the file from the official package?

Code: 
root@mowa219-gjp4-transcend-freebsd:~ # uname -vmKU
FreeBSD 14.1-RELEASE-p2 releng/14.1-n267684-1eba659e2f68 GENERIC amd64 1401000 1401000
root@mowa219-gjp4-transcend-freebsd:~ # file /boot/kernel/vmm.ko
/boot/kernel/vmm.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), BuildID[sha1]=8de9036e4e21cadd8e873e295424241b4c2bf65c, not stripped
root@mowa219-gjp4-transcend-freebsd:~ # size /boot/kernel/vmm.ko
    text    data       bss       dec        hex   filename
  181732   31048   3186760   3399540   0x33df74   /boot/kernel/vmm.ko
root@mowa219-gjp4-transcend-freebsd:~ # pkg which /boot/kernel/vmm.ko
/boot/kernel/vmm.ko was installed by package FreeBSD-kernel-generic-14.1p2
root@mowa219-gjp4-transcend-freebsd:~ #

I can provide a link to a copy.
 
cracauer@ said:
FreeBSD-update has nothing to do with source code.
Click to expand...

I want to upgrade 14.0 to 14.1 using freebsd-update,but if I do this directly,the recompiled vmm.ko will not work (already tried). I imagine that to make it work I need to modify its code (for sure correctly) before to launch the update script.
 
Cath O'Deray said:
ziomario would you like to try a copy of the file from the official package?

Code: 
root@mowa219-gjp4-transcend-freebsd:~ # uname -vmKU
FreeBSD 14.1-RELEASE-p2 releng/14.1-n267684-1eba659e2f68 GENERIC amd64 1401000 1401000
root@mowa219-gjp4-transcend-freebsd:~ # file /boot/kernel/vmm.ko
/boot/kernel/vmm.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), BuildID[sha1]=8de9036e4e21cadd8e873e295424241b4c2bf65c, not stripped
root@mowa219-gjp4-transcend-freebsd:~ # size /boot/kernel/vmm.ko
    text    data       bss       dec        hex   filename
  181732   31048   3186760   3399540   0x33df74   /boot/kernel/vmm.ko
root@mowa219-gjp4-transcend-freebsd:~ # pkg which /boot/kernel/vmm.ko
/boot/kernel/vmm.ko was installed by package FreeBSD-kernel-generic-14.1p2
root@mowa219-gjp4-transcend-freebsd:~ #

I can provide a link to a copy.
Click to expand...

sure. But do you think that it will work ? I see that the official vmm.ko modules don't have the working scheme inside the code. At least,not yet.
 
ziomario said:
I want to upgrade 14.0 to 14.1 using freebsd-update,but if I do this directly,the recompiled vmm.ko will not work
Click to expand...

There is no recompilation when you use freebsd-update.

I recommend that you:
- update to 14.1 from source (not freebsd-update)
- try it
- take a patched vmm.ko from somebody either as a source code patch or as a binary module
- try that

Myself I am not convinced of the string theory. But I have to wait to tomorrow to find out.
 
cracauer@ said:
Myself I am not convinced of the string theory. But I have to wait to tomorrow to find out.
Click to expand...

Me neither. I think that the string "KVMKVMKVM" should be added,but for sure it is not the only modification that should be added. Usually is not so easy. For sure there are more code to add. Maybe we are only seeing the tip of a small iceberg.
 
ziomario said:
Me neither. I think that the string "KVMKVMKVM" should be added,
Click to expand...
I don't understand you -- that's what I provided to you. Module that is for 14.1 that has this string in. Why didn't you tested it ? While it's not a solution to anything it's the essence of this thread.

Edit: you just pasted the results as I pasted this. :) Keeping it here for consistency.
 
_martin said:
One of the register used in the 14.1 function we are binary patching is different ( r14 vs r8). Last night I just patched it without thinking about it.

Now this module gets it right:
Code: 
Hypervisor: Origin = "KVMKVMKVM"
Now again, this is just to give you an idea what was changed in the module you provided. You could easily just change the bhyve_id in the module and compile that to achieve the same thing. And as I mentioned above it would be lucky cheap hack if that even works.

So again, this is not a solution to anything. You can try if you can fool the pass-through with this.
Click to expand...

The _martin hack worked. Houston,we have the passthru working on FreeBSD 14.1 + WIndows 11 with recent nvidia drivers installed. I hope that now you want to write a small tutorial to explain what to do to patch it and to patch the future versions of FreeBSD,14.2,14.3 and so on. Very thanks to _martin.

Code: 
[root@marietto /home/marietto]==> uname -a
FreeBSD marietto 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 releng/14.1-n267685-dcdea9e8623e GENERIC amd64

[root@marietto /home/marietto]==> kldstat | grep vmm
13    1 0xffffffff827ac000   37c9e8 vmm-141.ko
 
So,now,some developer who works on the virtualization area should give an answer to this question : why the string is not present on the vmm.ko source code of FreeBSD ? It seems relevant. It enables a very nice and useful function. It fixes the error 12 and / or 43 given by Windows maybe because it is not able to recognizes the nvidia drivers,because they refuse to be installed correctly without that string.
 
ziomario said:
The _martin hack worked.
Click to expand...
So I binary patched the module just so it uses the "KVMKVMKVM" string. As mentioned few times already though, that's not a solution but a hack. You could modify the source I mentioned above and have your own module without relying on somebody "from internet".

One could (not me;) ) check if bhyve_id was ever something else or not. If not it's clear you have been given a module that somebody changed (or compiled with different bhyve_id). Why? Well, maybe because he/she found out it works this way.

This trick reminds me of a joke where one expect when .doc file is renamed to .mp3 that it will read out the text. :).

Because it works it means the code within bhyve works. So why the change in the first place? Maybe some other part of the code is not yet finished or it's a bug. It may be worth asking in PR about this.
 
You must log in or register to reply here.
Back
Top