Solved Bhyve Windows 10 guests not booting anymore

[jardows]

I run some Windows 10 guests in bhyve on a couple of desktop computers. I don't use them much, they are more there if I need Windows apps than anything. I noticed this on one computer last week, and was thinking it was just come corruption with Windows. Today, I was testing another computer, and it is doing the same thing. When the VM starts, I open up a VNC viewer window, and it start through the initial boot screen, then gives me a "Preparing Automatic Repair" then blue screens with a stop code "Memory Management." My Xubuntu and Haiku VMs are loading without issue.

The one system is running FreeBSD 13.2-RELEASE, the other I just updated to 14.0-RC4. I am using /sysutils/vm-bhyve/ to manage the VMs, and have setup nvme emulation for the storage. I can try setting up a new VM or re-installing Windows in the VM to resolve the issue, but I really want to know what changed on both systems to cause the same error. Any info you can pass along?

 

[arken]

Good day!
I have similar problem with windows guests(Win7 and Win 10). You can try following steps:
1. Please add bellow string in each *.conf for UEFI guest
bhyve_options="-A"
2. Downgrade package edk2-bhyve to version 202202_10

In my case, with package version edk2-bhyve-g202308_3, I have the following problems:
1. The guest with Windows 7 stops at the loading stage.
2. The guest with Windows 10 boots, but there are problems with the hardware (serial ports). Error code 12.
I tested on versions 13.2 and 14.0-RC4

This issue was discussed here:

273560 – sysutils/edk2: bhyve flavor causes UEFI VMs to fail to boot after update to 202308

bugs.freebsd.org

Perhaps the problem is not limited to FreeBSD. Similar issues have been discussed here.

2241388 – Existing OVMF_VARS_4M.secboot.qcow2 UEFI VMs fail to boot with edk2 20230825-16

bugzilla.redhat.com

 

[jardows]

Some progress. I was able to downgrade the firmware on my desktop that has 13.2 installed, and get it to boot into Windows successfully. Just have to watch out for this one if I want to upgrade to 14.0.

I'm not sure how best to go about downgrading the firmware on my laptop, running 14.0-RC4. The 202202_10 version of the firmware was previously installed/built for 13.2, so I cannot install the downgrade from the pkg cache.

Oddly enough, I checked on my home desktop last night, which I had also upgraded to 14.0-RC4 over the weekend, and my Windows VM there started without any issue. Not sure what made the difference.

EDIT: I just read through the bug report, and added bhyve_options="-A" to the end of the configuration file, and that worked. Much simpler than trying to downgrade the firmware.

 

[marstr]

Some progress. I was able to downgrade the firmware on my desktop that has 13.2 installed, and get it to boot into Windows successfully. Just have to watch out for this one if I want to upgrade to 14.0.

I'm not sure how best to go about downgrading the firmware on my laptop, running 14.0-RC4. The 202202_10 version of the firmware was previously installed/built for 13.2, so I cannot install the downgrade from the pkg cache.

Oddly enough, I checked on my home desktop last night, which I had also upgraded to 14.0-RC4 over the weekend, and my Windows VM there started without any issue. Not sure what made the difference.

You can use the bhyvectl command to downgrade the firmware. This is a simpler process, but it is not as flexible as building the firmware from source.

 

[Charlie_]

On my 14.0-RC3 I was able to build g202202 with make FLAVOR=bhyve at https://cgit.freebsd.org/ports/tree/sysutils/edk2?id=3d9a815d9c5acbb71f4bb07738bdeab4879feacb.

 

[arken]

Some progress. I was able to downgrade the firmware on my desktop that has 13.2 installed, and get it to boot into Windows successfully. Just have to watch out for this one if I want to upgrade to 14.0.

I'm not sure how best to go about downgrading the firmware on my laptop, running 14.0-RC4. The 202202_10 version of the firmware was previously installed/built for 13.2, so I cannot install the downgrade from the pkg cache.

Oddly enough, I checked on my home desktop last night, which I had also upgraded to 14.0-RC4 over the weekend, and my Windows VM there started without any issue. Not sure what made the difference.

EDIT: I just read through the bug report, and added bhyve_options="-A" to the end of the configuration file, and that worked. Much simpler than trying to downgrade the firmware.

It seems to me that it depends on the processor model. The problem appears on older models. Could you provide information on the processor model on your work and home computer?
I'm using old notebook ThinkPad T530.
CPU: Intel(R) Core(TM) i5-3360M CPU @ 2.80GHz (2793.76-MHz K8-class CPU)

 

[rootbert]

that's what I have encountered also within the last few days. It depends on the CPU model. For example, on a Xeon CPU E3-1260L v5 with nvme I get a bluescreen (stop code MEMORY MANAGEMENT) when trying to use the nvme option whereas it is fine when using virtio. However, using a Xeon CPU E5-1630 v4 works fine with both configurations. Option -A seems to work for the E3-1260L

 

[jardows]

It seems to me that it depends on the processor model. The problem appears on older models. Could you provide information on the processor model on your work and home computer?
I'm using old notebook ThinkPad T530.
CPU: Intel(R) Core(TM) i5-3360M CPU @ 2.80GHz (2793.76-MHz K8-class CPU)

That makes some sense. My laptop is a 6th Gen i7, the work desktop is an 8th gen i7. My home desktop is a Ryzen 5600.

 

[arken]

This is weird. If the processor/architecture is supported by the operating system, then the behavior should be the same.
In addition to problems with starting the virtual machine, I have a problem with the emulated hardware inside the virtual machine.
The same virtual machine with Windows 10. Only the versions of the edk2-bhyve package differ.
edk2-bhyve-g202202_10 - device_manager_old.png
edk2-bhyve-g202308_3 - device_manager_new.png
There are no errors in the vm-bhyve log. Debugging is enabled.
Where to look for a possible error?