So I sat up half an evening working with Claude and ChatGPT on this, and this was the short transcript of the conversation. For reference. I was trying trying to run a Windows Server 2025 guest inside a FreeBSD 15.1-RELEASE-p2 machine. I used a raw file as the backing for the vm image (on a ZFS stripe of 4 x consumer SSDs) with a recordsize=32k. The bhyve script I used:
bhyve -c sockets=1,cores=1,threads=1 \
-m 20G \
-w -H -u \
-s 0,hostbridge \
-s 1,lpc \
-s 2,virtio-net,tap4,mac="${mac}" \
-s 3,nvme,/kvm/"${vm}"/disk0,nocache \
-s 4,virtio-rnd \
-s 29,fbuf,tcp=10.11.12.134:5900,w=1024,h=768,wait -s 30,xhci,tablet \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/kvm/"${vm}"/vars.fd,fwcfg="qemu" \
"${vm}"
The problem I had was that the host CPU was always relatively high compared with the guest (~5%)
1 vCPU 20%
2 vCPU 90%
4 vCPU 160%
8 vCPU 300%
Here's the transcript, I hope it's accurate and perhaps useful to someone in the bhyve team:
Key conclusion reached:
- The bhyve investigation established repeated guest reads of virtual HPET register 0xFED000F0, going through vmm_emulate_instruction()/vhpet_mmio_read().
- The Windows physical HPET device was disabled and BCD showed useplatformclock No, but the bhyve virtual HPET traffic remained at roughly 9,000–9,600 reads/sec.
- The proposed Server 2025 bhyve command uses UEFI, virtio networking, NVMe storage, VNC framebuffer, xHCI tablet, and 20 GB RAM; two vCPUs were suggested as an optional improvement over one.
Important DTrace evidence:
- vhpet_mmio_read at 0xFED000F0 repeatedly occurred.
- One sample showed 0xFED000F0 = 53,770 reads, versus 0xFED00100 = 2,711 and 0xFED00108 = 2,711.
- Another sample showed roughly 9,000–9,600 reads/sec at 0xFED000F0.
- Stack traces showed vmm_emulate_instruction -> vm_run and vhpet_mmio_read.
- vmm_emulate_instruction received arg1=0xfed000f0.
- FreeBSD source showed bhyve's virtual HPET is exposed at VHPET_BASE = 0xFED00000 and advertised through ACPI.
Windows checks:
- High precision event timer ACPI\PNP0103\0 was initially OK.
- It was disabled and subsequently reported Status Error.
- bcdedit /enum "{current}" showed useplatformclock No.
- The intention was to reboot and then move on rather than patch bhyve.