bhyve Bhyve can't pass thru any of my NVIDIA graphic cards to an Ubuntu bhyved os : the vm freezes before recognizing the disk

ziomario said:
Bhyve is not yet mature,at least on FreeBSD. It needs more development.
Click to expand...
You could say that about most any software. All this is subjective. bhyve does what its users need.
This virtual machine GPU passthru is not really a measurement of the maturity of this software.
GPU passshtru to a foreign VM is the everest of virtual machining.
 
In my opinion,it's not subjective. If u take in consideration qemu / kvm on Linux,you will see that it is more mature than bhyve on FreeBSD. For sure,in the next future bhyve will be more mature and it will support more features. Even the passthru will work,but for the moment,it is not perfect. According to the web page that I have shown,it seems that the bhyve passthrough works better on omnios / illumos. So,it's not a subjective matter,but these are facts.
 
Theoretically the FreeBSD developers can give a look at the source code of bhyve installed on OmniOS,to understand how the OmniOS deveopers have fixed the problems related to the gpu pass thru ?
 
ziomario said:
what I did is easy. I've got this file : FreeBSD-14.0-CURRENT-amd64-20210805-f3a3b061216-248478-disc1.iso and I've dd it on the usb stick. It detected and configured correcrtly the network but it is not able to grab any file. what u suggest me to do ?
Click to expand...
I wonder why the installer needed to fetch anything. Since FreeBSD-14.0-CURRENT-amd64-20210805-f3a3b061216-248478-disc1.iso is not a bootonly ISO, distfiles are found under /usr/freebsd-dist.
Can't advise much here as I usually do the installation manually. Which consists of little more than extracting distfiles into the installation root.
 
ziomario said:
because I read that on OmniOS the bhyve passthru works,according with this post
Click to expand...
In FreeBSD bhyve passthrough works as well. I've been using it for a couple of years now, as do many others. To be precise, it just doesn't work smoothly in ALL cases, and your case seems to be a bit of a bad luck. What else? This: GPU passthrough is a relatively new thing in bhyve -- you may have missed that in the documentation you read in a hurry.
As for OmniOS and the probability that its port of bhyve will work better for your case than FreeBSD... I'll be glad if it does :).

However. OmniOS is Illumos-based (tracing back to OpenSolaris). Developer resource is, I suppose, far more limited than that of FreeBSD. But bhyve code comes from FreeBSD, not from anything Solaris. Were they able to solve all the passthrough problems it has under FreeBSD?? I think it more likely that it works in OmniOS as much as it does in FreeBSD. But perhaps it's worthy trying.
 
free-and-bsd said:
I wonder why the installer needed to fetch anything. Since FreeBSD-14.0-CURRENT-amd64-20210805-f3a3b061216-248478-disc1.iso is not a bootonly ISO, distfiles are found under /usr/freebsd-dist.
Can't advise much here as I usually do the installation manually. Which consists of little more than extracting distfiles into the installation root.
Click to expand...

The error occurs if base-dbg or lib32-dbg is selected. At the beginning I included the base file (dbg) because I've thought that they,as a base files,cannot be deselected,otherwise FreeBSD couldn't be installed. But then,I tried to ignore them and the installer started and...the odd thing is that the installer installed that base files anyway even if I had deselected it.
 
free-and-bsd said:
In FreeBSD bhyve passthrough works as well. I've been using it for a couple of years now, as do many others. To be precise, it just doesn't work smoothly in ALL cases, and your case seems to be a bit of a bad luck. What else? This: GPU passthrough is a relatively new thing in bhyve -- you may have missed that in the documentation you read in a hurry.
As for OmniOS and the probability that its port of bhyve will work better for your case than FreeBSD... I'll be glad if it does :).

However. OmniOS is Illumos-based (tracing back to OpenSolaris). Developer resource is, I suppose, far more limited than that of FreeBSD. But bhyve code comes from FreeBSD, not from anything Solaris. Were they able to solve all the passthrough problems it has under FreeBSD?? I think it more likely that it works in OmniOS as much as it does in FreeBSD. But perhaps it's worthy trying.
Click to expand...

At the end of the game I've passed thru my RTX 2080 ti graphic card to Ubuntu 21.04 vm with the great help of a developer and it can works even on Windows 10. I will check later. He applied the patches that he wrote on my FreeBSD 13R and the passhtru started working. So now I'm happy. Instead,OmniOS freezed when I was playing with it. At this point I don't think that I will play with it anymore. I have everything I need on FreeBSD. You say that the GPU passhtru works good for many. I really don't know. Maybe for the old graphic cards. From what I see,the newest cards have a bug that still needs to be fixed,called "bar registration" or something like this.
 
After looking on the wiki for a bit, I'm thinking that bhyve itself is the issue - IIRC, VirtualBox can access the GPU on the host, with some help. I know VMWare can (but requires a special driver/plugin). bhyve is just not that advanced. So I think you just flat out are expecting too much out of bhyve.
 
ziomario said:
You can google a little bit and find people who says that the GPU passthrough works good for them.
Click to expand...
Yes, Phishfry here and one guy on twitter. That's exactly two accounts in favor of it working. All other evidence suggests otherwise.

ziomario said:
what's the difference between the bhyve passtrhu and the gpu passthru on bhyve ?
Click to expand...
GPU passthrough needs to do some kind of magic with the video card bios initialization. It's not really documented anywhere and very few people know the details.
 
astyle said:
After looking on the wiki for a bit, I'm thinking that bhyve itself is the issue - IIRC, VirtualBox can access the GPU on the host, with some help. I know VMWare can (but requires a special driver/plugin). bhyve is just not that advanced. So I think you just flat out are expecting too much out of bhyve.
Click to expand...

Virtualbox and vmware workstation don't support gpu passthrough. ESXI does it,but it is not a type 2 hypervisor. I want this kind of hypervisor. Anyway,it does not work well with my hardware.
 
shkhln said:
Yes, Phishfry here and one guy on twitter. That's exactly two accounts in favor of it working. All other evidence suggests otherwise.


GPU passthrough needs to do some kind of magic with the video card bios initialization. It's not really documented anywhere and very few people know the details.
Click to expand...
So I've been lucky. The GPU passthrough with Bhyve is working for me on FreeBSD as host and Ubuntu guest and it almost work for Windows 10. I need only to fix the error 43,but I know how to do that. At this point I'm satisfied,because I can use FreeBSD for everything and I can say bye to Linux as primary OS,since during the years it became excessively and unnecessarily complicated.
 
Now I'm working on the configuration of the GPU passthrough on Windows 10. I see that anydesk does not work well. Most of the time it won't connect to Windows 10 VM. I would like to know if you know an alternative remote desktop manager which works natively on FreeBSD and on Windows 10. Maybe I can install VNC server on Windows 10 and vncviewer on FreeBSD. I wanna check if the cause of the problem is anydesk or something else.
 
ziomario said:
what's the difference between the bhyve passtrhu and the gpu passthru on bhyve ?
Click to expand...
I can also add that 'bhyve passthrough' is a general technology allowing to passthrough PCI devices to bhyve VM. GPU originally was NOT among such devices, nor was it, I understand, particularly targeted as such. Use cases include passing through storage devices, USB devices, NICs.

In my reading about bhyve / virtualization general use case, I don't find GPU passthrough listed among solutions that really make much sense.
What advantage is there in using GPU passthrough with VM over using your GPU directly in your OS? In the context of virtualization technology there seems to be few, if any at all. Yes, you CAN, theoretically, insert several GPU cards into your machine and even make them WORK simultaneously. If you find a motherboard supporting this and a processor powerful enough to make it worth the while... Suppose it all done succesfully -- what do you really gain? Granted, there MAY be some exceptional cases, but not much more.

So I, honestly, think even you, after you 'master' this case of yours to your own satisfaction, will find it of little practical value. Other than satisfaction from the fact that it IS possible, of course. This type of satisfaction I completely understand as I've done the same many times :) :) :) That's why I also tried to help as much as I could.
 
Because the nvidia driver written for FreeBSD does not support the hardware acceleration natively,I read. And FreeBSD does not support the CUDA libraries,also. Instead,all of this is supported inside a Windows or Linux OS. Since I'm also a blender / CG tools user,I can take advantage of the GPU hardware acceleration to render faster my CG projects.
 
ziomario said:
Because the nvidia driver written for FreeBSD does not support the hardware acceleration natively,I read. And FreeBSD does not support the CUDA libraries,also. Instead,all of this is supported inside a Windows or Linux OS. Since I'm also a blender / CG tools user,I can take advantage of the GPU hardware acceleration to render faster my CG projects.
Click to expand...
Sorry, I'm with Team Red. But I do get the appeal of hardware acceleration under FreeBSD.
 
ziomario said:
So I've been lucky. The GPU passthrough with Bhyve is working for me on FreeBSD as host and Ubuntu guest
Click to expand...
Would you mind running glxinfo and nvidia-smi on Ubuntu?

ziomario said:
Because the nvidia driver written for FreeBSD does not support the hardware acceleration natively,I read.
Click to expand...
That a bit too broad of a statement. OpenGL obviously requires hardware acceleration, Vulkan does as well.

ziomario said:
Since I'm also a blender / CG tools user,I can take advantage of the GPU hardware acceleration to render faster my CG projects.
Click to expand...
Is this all about Blender?
 
shkhln said:
That a bit too broad of a statement. OpengGL obviously requires hardware acceleration, Vulkan does as well.
Click to expand...
I've done OpenGL programming myself back in its heyday, so I know for a fact that it does not require hardware acceleration. It does have API's to take advantage of hardware acceleration, if compatible hardware is available. Vulkan is the same - you can use it to draw circles in all colors of the rainbow, anywhere on the screen, and make it jump around - but 'hardware acceleration in Vulkan' is simply possible via a C/C++ API that you use after checking for compatible hardware.
 
I'm not so experienced. I don't know anything about OpenGL and Vulkan. But I know that I need to run Linux (or Windows) + Nvidia driver + Cuda libraries if I want to take advantage of the full power of my GPU and use it with Blender for rendering with cycles,but not only in Blender,there are some other CG + graphic tools that need them. At the moment my Ubuntu VM is damaged. For some reason I'm not able anymore to login inside the VM. When I click on the Accedi / Login button,I get bounced cyclically on the same login page. I've created a short video to show you what happens :

https://drive.google.com/file/d/1wfmdd_rA2NqgIj6DzDAVRc6yjHakn1zh/view?usp=sharing

The last step that I did before to become unable to continue the work has been the installing of the CUDA libraries. If I'm not able to understand the reason of the odd behavior,I should recreate a new ubuntu VM from scratch. Instead,regarding Windows 10,I'm fighthing with the error 43. I shouldn’t get it,because NVIDIA told that from the Driver 465.89 it supports the virtualization,but in my specific case it seems not true. You can read by yourself what they said :

https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/geforce-gpu-passthrough-for-windows-virtual-machine-(beta)
 

Attachments

  • Screenshot_20210830_140107.png
    Screenshot_20210830_140107.png
    531.5 KB · Views: 198
  • 56f67c8ed5b1a5bfebb738be4a6c0c4887b00175.jpeg
    56f67c8ed5b1a5bfebb738be4a6c0c4887b00175.jpeg
    121.8 KB · Views: 175
You must log in or register to reply here.
Back
Top