I have been running quite a few windows 11 virtual machines on a couple of servers using BHYVE and 14.2-RELEASE.
Upon upgrading win11 to 24H2 the host cpu usage at idle (4 cpu guest) has increase from ~2% to %20 of a CPU (based on top). This has happened on both
intel and AMD CPUs. The other item of note is that the number of voluntary context switches for the BHYVE process has increased from approx 330/sec to 12000/sec.
As part of testing, I have created a fresh windows install (win11 21H2), then updated to 23H2, then 24H2. There was a slight jump to 23H2, but the huge jump was upon updating to 24H2.
The other thing to note is that it is proportional to the number of virtual CPUs. i,e.
These numbers are not precise, but a pattern is apparent. The above was done on a 8 core Ryzen, but similar occurs on xeon and threadripper.
I have tried disabling many different windows devices/functions but nothing yet seems to make any difference.
The system cpu used by interrupts on the guest has also increased from 0% to ~1.5-2%.
I am using vm-bhyve to manage the vms.
Config for the test VM is:
I am at a bit of a loss and if anyone can confirm my findings, and suggest diagnostic methods, I would greatly appreciate any help.
regards
Duncan
Upon upgrading win11 to 24H2 the host cpu usage at idle (4 cpu guest) has increase from ~2% to %20 of a CPU (based on top). This has happened on both
intel and AMD CPUs. The other item of note is that the number of voluntary context switches for the BHYVE process has increased from approx 330/sec to 12000/sec.
As part of testing, I have created a fresh windows install (win11 21H2), then updated to 23H2, then 24H2. There was a slight jump to 23H2, but the huge jump was upon updating to 24H2.
The other thing to note is that it is proportional to the number of virtual CPUs. i,e.
Nr CPUs | CPU usage (percentage of 1 core) | VCSW per second |
1 | 4% | ~500 |
2 | 9% | ~3000 |
3 | 15% | ~7000 |
4 | 20% | ~12000 |
6 | 27% | ~19000 |
8 | 36% | ~27000 |
These numbers are not precise, but a pattern is apparent. The above was done on a 8 core Ryzen, but similar occurs on xeon and threadripper.
I have tried disabling many different windows devices/functions but nothing yet seems to make any difference.
The system cpu used by interrupts on the guest has also increased from 0% to ~1.5-2%.
I am using vm-bhyve to manage the vms.
Config for the test VM is:
Code:
loader="uefi"
bhyve_options="-A"
graphics="yes"
#graphics_res="1920x1080"
graphics_res="1280x1024"
graphics_port="5951"
vnc_password="********"
xhci_mouse="yes"
# If not specified, cpu=n will give the guest n discrete CPU sockets.
# This is generally OK for Linux or BSD guests, but Windows throws a fit
# due to licensing issues, so we specify CPU topology manually here.
cpu_sockets=1
cpu_cores=4
cpu=4
# Remember, a guest doesn’t need extra RAM for filesystem caching--
# the host handles that for it. 4G is ludicrously low for Windows on hardware,
# but it’s generally more than sufficient for a guest.
memory=4G
wired_memory="no"
# put up to 8 disks on a single ahci controller. This avoids the creation of
# a new “controller” on a new “PCIe slot” for each drive added to the guest.
ahci_device_limit="8"
# e1000 works out-of-the-box, but virtio-net performs better. Virtio support
# is built in on FreeBSD and Linux guests, but Windows guests will need
# to have virtio drivers manually installed.
# network0_type="e1000"
network0_type="virtio-net"
network0_switch="lan"
# bhyve/nvme storage is considerably faster than bhyve/virtio-blk
# storage in my testing, on Windows, Linux, and FreeBSD guests alike.
disk0_type="nvme"
# disk0_opts="maxq=1024,qsz=1024,ioslots=16,sectsz=512,ser=A0000001,dsm=auto"
disk0_name="disk0.img"
# This gives the guest a virtual "optical" drive. Specifying disk1_dev=”custom”
# allows us to provide a full path to the ISO.
disk1_type="ahci-cd"
disk1_dev="custom"
disk1_name="/vm/.iso/virtio-win.iso"
# windows expects the host to expose localtime by default, not UTC
utctime="no"
uuid="d62e8914-2bf4-11f0-88fb-d05099d1a548"
network0_mac="58:9c:fc:04:96:56"I am at a bit of a loss and if anyone can confirm my findings, and suggest diagnostic methods, I would greatly appreciate any help.
regards
Duncan