bhyve Windows Server slow IO

Phishfry

Son of Beastie

Reaction score: 1,641
Messages: 4,706

I don't see the advantage of passing through an NVMe.

From my findings your better off hosting VM's on them and using ahci-hd for the images versus -s 7:0,nvme,/dev/nda2
 

Lamia

Well-Known Member

Reaction score: 72
Messages: 419

Vm-bhvye is very simplistic. CBSD is robust with support for multiple OS platforms and continues to require more testing from its developers to ensure wide use. Chyves is only for FreeBSD. This background is necessary because I spent the last five days trying to get bhyve run with no issues.

CBSD won't run ISOs from its own repo despite using 'cbsd bconstruct-tui' wizard. Vm-bhvye works with ease but later fails. I got latest Freepbx running on it and "vm console..." showed expected information while "cbsd blogin..." hangs just as it reads its CD, peripherals and boot entries.

The problem with vm-bhvye is that web access is lost after one or two clicks at the initial setup after a fully automatic installation over console.

The vm-public bridge like tap0 keeps changing its ether address at every reboot. What this means is that the [new] MAC address would not be the same as MAC address earlier enter into the VM-BHYVE_DIR/.config/system.conf in host machine AND the etc/sysconfig/network-script/ifcfg-eth0 in the VM itself. When the Ethernet address is the same and the Freepbx web UI is accessible, it is just for few seconds and clicks. Connection via browser will be lost in no time. Usually at this time, the VM could be pinged from outside it e.g. from other jails or host.
Even on creating a bridge connected to tap0 & vm-public and using its static ether address in the above files, the gateway and other IP address cannot be pinged. A passthru of ALL in system.conf did not work either.

Any suggestions would be appreciated.

It is not intended to hijack this thread; it is only coincidental with almost the same problem.
 

Lamia

Well-Known Member

Reaction score: 72
Messages: 419

Vm-bhvye is very simplistic. CBSD is robust with support for multiple OS platforms and continues to require more testing from its developers to ensure wide use. Chyves is only for FreeBSD. This background is necessary because I spent the last five days trying to get bhyve run with no issues.

CBSD won't run ISOs from its own repo despite using 'cbsd bconstruct-tui' wizard. Vm-bhvye works with ease but later fails. I got latest Freepbx running on it and "vm console..." showed expected information while "cbsd blogin..." hangs just as it reads its CD, peripherals and boot entries.

The problem with vm-bhvye is that web access is lost after one or two clicks at the initial setup after a fully automatic installation over console.

The vm-public bridge like tap0 keeps changing its ether address at every reboot. What this means is that the [new] MAC address would not be the same as MAC address earlier enter into the VM-BHYVE_DIR/.config/system.conf in host machine AND the etc/sysconfig/network-script/ifcfg-eth0 in the VM itself. When the Ethernet address is the same and the Freepbx web UI is accessible, it is just for few seconds and clicks. Connection via browser will be lost in no time. Usually at this time, the VM could be pinged from outside it e.g. from other jails or host.
Even on creating a bridge connected to tap0 & vm-public and using its static ether address in the above files, the gateway and other IP address cannot be pinged. A passthru of ALL in system.conf did not work either.

Any suggestions would be appreciated.

It is not intended to hijack this thread; it is only coincidental with almost the same problem.
It's now fixed. Passthru finally worked. Existing documentation is not very clear on where changes are required. I needed to make changes to both the loader.conf and system.conf. Then it goes beyond that to systonically bridge two interfaces in a VM.


In fact, this is just another form of hack that makes me think that FreeBSD is not for the faint-hearted.
 

Phishfry

Son of Beastie

Reaction score: 1,641
Messages: 4,706

Does Xeon X56xx support this?
From what I read it seems that APIC virtualized(APCIv) was first a software feature that was moved onto the CPU.
So maybe E5 Xeon V2 family was the first with APCIv on the CPU.
 
OP
OP
stratacast1

stratacast1

Well-Known Member

Reaction score: 24
Messages: 286

I'm not sure if this helps anyone, but I had a need to run a Windows server VM again and my host hardware has changed. In the past, I was testing this on an i5 4670 with 8GB of RAM and on an SSD. Most everything worked great but it choked at IO. Unzip a 100kb file? Took 30 seconds. Nasty. I did it again today with the same files and it was instantaneous. Tried again with a 700MB 7z archive and it did it at the speed I expected. Very nice actually, I feel like I can actually recommend bhyve to people now for virtualizing Windows :)

New hardware: Ryzen 3600, 16GB RAM, running my VMs off a zpool that is on a Samsung 840 Pro SSD
OS: 12.1-RELEASE
 

zader

Active Member

Reaction score: 13
Messages: 136

I feel like I can actually recommend bhyve to people now for virtualizing Windows :)
sure wish pci passthrough worked better for graphics cards .. Id never look back if my 2080ti worked better :( on a windows vm .. perhaps I missed something..

did you end up passing the nvme? or did the new hardware fix the io? or you just created a pool/dataset for vm's on a new drive?
 
OP
OP
stratacast1

stratacast1

Well-Known Member

Reaction score: 24
Messages: 286

sure wish pci passthrough worked better for graphics cards .. Id never look back if my 2080ti worked better :( on a windows vm .. perhaps I missed something..

did you end up passing the nvme? or did the new hardware fix the io? or you just created a pool/dataset for vm's on a new drive?
The OS isn't running on an NVMe drive, I have it on a standard SATA SSD. Maybe the new hardware fixed IO? I guess since this is AMD it could behave differently than an Intel box. I'm using vm-bhyve so I just used what they have

Code:
loader="uefi"
graphics="yes"
xhci_mouse="yes"
cpu="3"
memory="6G"

# put up to 8 disks on a single ahci controller.
# without this, adding a disk pushes the following network devices onto higher slot numbers,
# which causes windows to see them as a new interface
ahci_device_limit="8"

# ideally this should be changed to virtio-net and drivers installed in the guest
# e1000 works out-of-the-box
network0_type="e1000"
network0_switch="public"

disk0_type="ahci-hd"
disk0_name="disk0.img"

# windows expects the host to expose localtime by default, not UTC
utctime="no"
uuid=""
network0_mac=""
 

m1001101

New Member


Messages: 10

For windows server, my recommendation config is

1. apply this commit https://reviews.freebsd.org/rS358848
2. using nvme as disk backend or passthough a nvme disk.
3. passthrough a nic instead of virtio-net
4. apply this commit https://reviews.freebsd.org/rS349184
5. apply this commit https://reviews.freebsd.org/rS348779

Windows will as fast as it could be.
I've Xeon E3 1220 and bhyve work very slow on 12.1.
To test the patches I've upgraded to CURRENT and now bhyve working great!
In order to maintain 12.1 is possible to apply these patches and build only bhyve and vmm?
Tested copying the new files in /usr/src/sys/usr.sbin/bhyve and make, now I have the executable in /usr/obj/. How I can install? Manually move the executable and libraries? And for vmm?

Thanks
 

Zirias

Aspiring Daemon

Reaction score: 252
Messages: 650

In order to maintain 12.1 is possible to apply these patches and build only bhyve and vmm?
Please see here: https://forums.freebsd.org/threads/bhyve-windows-server-slow-io.71199/post-456220 -- it only contains two of those patches, but I found the first one is the one making the huge difference for windows guests.

About only building parts of the system, it definitely works for bhyve (just use make in the appropriate source directory), I didn't try to just build the vmm module but built a complete kernel, so maybe this works as well.
 
Top