libvirt with bhyve

Phishfry

Phishfry

I have been learning virtulization and I started with Xen Dom0 on FreeBSD.
Got it working but I really got stuck until I figured out that libvirt naively compiles with support for bhyve. I had to build from ports with Xen option enabled. Then everything sprang to life. Boy I hope that hint helps somebody. I saw no such information anywhere.

Well I put Xen VM head to head with bhyve VM and test storage and speeds. My NVMe array in Xen was only giving 10% of the speed of the hypervisor. On bhyve I saw maybe 10% loss at max over hypervisor speeds. So I ditched Xen.

When I ditched Xen I still had deskutils/virt-manager installed on the hypervisor with a small xorg install. I was messing with it as it seems tied to libvirt and a GUI for VM's is nice.

So my question is: Does bhyve require libvirt? I know with bhyve you can pass disks through as AHCI or libvirt disk.
What is recommended? It seems that libvirt itself can do some virtualization stuff.

I am having a hard time understanding when to use it. How about speed. Are its disk and network drivers OK?
I could pass through a whole extra ethernet interface versus libvirt one.

Please shine some light on the subject for me. libvirt has a driver substructure and has overlapping virtualization features too.
Is it just a helper application/library?
 
So my question is: Does bhyve require libvirt? I know with bhyve you can pass disks through as AHCI or libvirt disk.
Click to expand...

There's a bit of confusion here.

bhyve by itself has nothing to do with libvirt and does not require it.

libvirt is a framework which provides a standard interface layer between hypervisors (kvm/bhyve/xen/etc) and applications. libvirt for example has a standard "start a virtual machine" function which something like deskutils/virt-manager can call. libvirt then runs the correct commands for the relevant hypervisor. This allows applications such as virt-manager to support all the hypervisors that libvirt supports without having to have specific support built-in for each.

Is it just a helper application/library?
Click to expand...

Yes, it has nothing to do with the actual running of the guest and really just provides a management interface.

VirtIO on the other hand is a group of devices designed for virtualisation. These include virtio-net, virtio-blk (block device), among others. Most hypervisors provide support for these devices as they tend to perform much better than emulating real hardware. Bhyve for example supports e1000 and virtio-net for network devices. e1000 tries to completely emulate an Intel network adapter, whereas virtio-net is specifically designed as a virtual interface.

You generally always want to use the VirtIO devices instead of ahci/e1000/etc as emulating real hardware is a bit of a hack really.
 
Sort of similar

If you're using bhyve and not using the UEFI/VNC support, then you will usually access the guest using the serial port com1. You can have com1 directly connected to your terminal, but that means the guest runs in the foreground and you can't get back to the host cli until you shut the guest down (or open another terminal/ssh). It's common to connect com1 to a nmdm device, which connects the com port to a device node rather than your terminal. You can then connect to that device node with a terminal emulator (such as cu) to communicate with the guest.

nmdm is a null modem driver in FreeBSD, so basically you're effectively connecting the guest com1 port to a serial modem cable, which is exposed on the host as /dev/nmdmX.

virtio-console is a VirtIO device that allows providing additional serial ports to allow communication between the host and guest. For example, FreeNAS uses this feature to provide a communication channel between the two so it can receive information from the guest and send commands in, etc.
 
usdmatt said:
You generally always want to use the VirtIO devices instead of ahci/e1000/etc as emulating real hardware is a bit of a hack really.
Click to expand...
I missed that line. I would have thought that passthrough was better.
Did have some problems with an Intel/SuperMicro Quad NIC.82576 showing igb.
It did not like 2 of the subnodes. 132/0/0 worked fine but 132/0/1 did not. 134/0/0 worked 134/0/1 did not.
So the e1000 driver does seem temperamental.
I ended up with two em interface Quad NICs Intel/IBM 82572 using em driver.
With this I have solid 8 lines of individual VM's right to my physical switch. Cheap quad cards I had stashed from 2012.

I did find that Vale was simpler than I imagined to get working thanks to Jim@netgate gist notes.

Thanks usdmatt for your help. I really respect the work you have done on vm-bhyve.
It seems to be the prevalent tool I see mentioned.
 
I missed that line. I would have thought that passthrough was better.
Click to expand...

Yes, I would expect pass-through to be the best performing option as the VM has direct access to a real network card, although you have to give up the interface* to the guest. My line above was really just in relation to "virtual" devices, where virtio-net is a much better option compared to a fully software-based device which is pretending to be hardware.

*Of course some of the higher end network cards now support SR-IOV, so a single PCI-e device can appear as multiple devices, which can then be passed to guests. Maybe I'll get to try that out one day...
 
You must log in or register to reply here.
Back
Top