Solved Can't boot FreeBSD 15.0-RELEASE guest

tembun

tembun

I'm trying to run a VM with FreeBSD 15.0-RELEASE as guest and FreeBSD 15.1-STABLE as host. I'm using sysutils/vm-bhyve.

Here's my config for the FreeBSD guest VM:

Code: 
loader="bhyveload"
cpu=2
memory=4G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="31c6b3d1-61b3-11f1-ac1e-9c2dcd4af482"
network0_mac="58:9c:fc:0a:ee:6b"

I installed the guest OS:
Code: 
vm install -f freebsd150 FreeBSD-15.0-RELEASE-amd64-memstick.img

and it worked just fine.

During the installation, I chose ZFS configuration with GPT (BIOS). All other ZFS options were default.

But after finishing the installation, I can't boot into the guest:
Code: 
# vm start -f freebsd150

Starting freebsd150
  * found guest in /usr/local/vm/freebsd150
  * booting...
Consoles: userboot  

FreeBSD/amd64 User boot lua, Revision 3.0
(Fri May 29 13:05:48 MSK 2026 tem@rio)
ERROR: cannot open /boot/lua/loader.lua: no such file or directory.

Before quitting the installer, I opened the shell in the newly installed system and verified that /boot/lua/loader exists:

Code: 
# ls -l /boot/lua/loader.lua 
-r--r--r--  1 root wheel 2364 Nov 28  2025 /boot/lua/loader.lua

What may be the problem?
 
Assuming the boot menu is displayed, "3. Escape to loader prompt" and check "currdev" (enter show currdev). It should look like: currdev=zfs:zroot/ROOT/default:.
 
T-Daemon said:
Assuming the boot menu is displayed, "3. Escape to loader prompt" and check "currdev" (enter show currdev). It should look like: currdev=zfs:zroot/ROOT/default:.
Click to expand...
Oh, no, it is set to disk0p3:
Code: 
show
LINES=24
autoboot_delay=NO
bhyve_vm_name=freebsd150
boot_serial=1
console=userboot
currdev=disk0p3:
interpret=OK
loaddev=disk0p3:
module_verbose=2
prompt=${interpret}
script.lang=lua
smbios.bios.vendor=BHYVE
smbios.system.uuid=31c6b3d1-61b3-11f1-ac1e-9c2dcd4af482
twiddle_divisor=16

At what part of installation might I miss something so that it's set to a wrong value?
 
tembun said:
At what part of installation might I miss something so that it's set to a wrong value?
Click to expand...
This shouldn't happen, normally. I can't tell why it happened. It doesn't happen in my test setup (host 15.0-RELEASE, guest vm-bhyve 15.0-RELEASE).

You can work around the issue by setting the "currdev" variable in /boot/loader.conf (see loader_simp(8)).

To get the system booted, use the set currdev=... command at the loader prompt.
 
T-Daemon said:
This shouldn't happen, normally. I can't tell why it happened. It doesn't happen in my test setup (host 15.0-RELEASE, guest vm-bhyve 15.0-RELEASE).

You can work around the issue by setting the "currdev" variable in /boot/loader.conf (see loader_simp(8)).

To get the system booted, use the set currdev=.. variable at the loader prompt.
Click to expand...
I entered the installer shell, mounted zroot under /tmp/mnt, and edited the /boot/loader.conf there:

Code: 
# cat /tmp/mnt/boot/loader.conf
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
zfs_load="YES"
currdev="zfs:zroot/ROOT/default:"

After that, unmounted /tmp/mnt and tried to start the VM again - same error: currdev still points to disk0p3:.

I have ZFS support compiled into the kernel on the host:
Code: 
# kldstat -m zfs
Id  Refs Name
 14    1 zfs
 
Emrion said:
Use UEFI as boot method for this VM, and install it anew. You might have less problem.
Click to expand...
I already tried that, but I faced another issue there:
Code: 
BdsDxe: failed to load Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x4,0x0): Not Found
BdsDxe: No bootable option or device was found.
BdsDxe: Press any key to enter the Boot Manager Menu.

I was trying to solve that yesterday for a couple of hours and the only thing that worked is:
Code: 
cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI

before the first boot of the VM. boot/efi/EFI/BOOT/BOOTX64.EFI, as I understood, is a reserved location for a backup loader, which is used when nothing else was found. It seemed like a weird kludge to me, it looks like a very basic setup (FreeBSD host + FreeBSD bhyve guest), but it doesn't work out of the box... So I decided to try it with BIOS and bhyveload(8), but it doesn't seem to work with defaults either (at least, on my machine; T-Daemon for instance said that he doesn't have this problem with default installation).
 
T-Daemon said:
It doesn't happen in my test setup (host 15.0-RELEASE, guest vm-bhyve 15.0-RELEASE).
Click to expand...
I just tried exactly this setup (both host and guest 15.0-RELEASE) on another machine - everything works fine. The only difference is that my current setup, I use UFS on the host and ZFS for the guest, whereas in the second setup, both guest and host use ZFS. I don't think that this is the cause of the problem though.
 
T-Daemon said:
I'll try the 15.1-STABLE branch on the host, and report back.
Click to expand...
No problem on 15.1-STABLE host either (ZFS boot environment).
Code: 
# uname -aKU
FreeBSD test 15.1-STABLE FreeBSD 15.1-STABLE stable/15-n283851-7546f06f5e06 GENERIC amd64 1501501 1501501
The vm-bhyve (loader="bhyveload") 15.0-RELEASE guest boots just fine, with currdev="zfs:zroot/ROOT/default: set automatically.

All host systems in my test setup have ZFS.
 
tembun said:
before the first boot of the VM. boot/efi/EFI/BOOT/BOOTX64.EFI,
Click to expand...
This is because the efi vars are not saved with the VM.
With vm-bhyve it's uefi_vars=true.

I can't understand that this is not the default...

My advice is to stick to UEFI. BIOS is the past, and it will less and less maintained and supported.
 
Emrion said:
This is because the efi vars are not saved with the VM.
With vm-bhyve it's uefi_vars=true.
Click to expand...
Yeah, I did have this in my VM config:
Code: 
uefi_vars=yes
but it still didn't work and terminated with the very same error.

And after I did boot/efi/EFI/BOOT/BOOTX64.EFI trick and finally booted into the VM, efi vars were saved - uefi_vars.fd (or something like that) appeared in the VM directory.

I will check this one one more time, maybe something was messed up that time.
 
tembun said:
Yeah, I did have this in my VM config:
Click to expand...
You need that before the first launch, at installation time.
When FreeBSD is installing (as other OSes) it sets one efi var to its own loader. If it's not set at this time, it's too late, nothing will be written later in the efi vars (unless you do it yourself of course).
 
It seems that I found the cause of the problem. I do build the world and the kernel from sources, and I compile only those modules that I actually use. As I mentioned above, my host system uses UFS, so I didn't have a ZFS support for the kernel and userland. And in src.conf(5) I had:
Code: 
WITHOUT_ZFS=YES
WITHOUT_LOADER_ZFS=YES

Recently I decided to learn more about ZFS and for that I decided to set up a VM, where I could play with it safely. I did recompile the kernel with options ZFS, but I neglected to remove WITHOUT_LOADER_ZFS and recompile the world. WITHOUT_ZFS wasn't the problem - it only affects userland utilities, but as you can see, it's important to have both ZFS support for the kernel and for the loader.

Now everything works fine for both BIOS and UEFI. Thank you for the help, mates!
 
You must log in or register to reply here.
Back
Top