Solved [bhyve] Newer zpool version in guest than supported by host

zirias@

zirias@

Developer
I have all my FreeBSD machines (metal and virtual) on -RELEASE except for one: my "testbuilder", used with poudriere to test ports. Ports must work with -CURRENT as well, so the poudriere host for this must run -CURRENT. It's a virtual machine in bhyve, using bhyveload(8) to boot, on a 13.1-RELEASE host.

Now, for the first time, I tried upgrading this machine's ZFS pool, and ended up with an unbootable machine (of course, I had a snapshot of its virtual disk). Seeing the virtual disk had a classic ("BIOS") boot partition, I then tried again, but this time changing the bhyve configuration to boot with "uefi-csm" and updating the MBR and boot partition with gpart bootcode. Again, it didn't boot.

Now, I have questions ?
  • Do I interpret it correctly that bhyveload(8) loads the kernel directly from the virtual disk, so can't possibly work if the zpool on this virtual disk uses a version it doesn't know yet?
  • If that's true, why doesn't booting using uefi-csm work either? :-/ Something I'm overlooking here?
Remarks: It's not really important for me, I'm just curious. Furthermore, I don't think it's a question about -CURRENT, but more generically a question about running a newer FreeBSD version in bhyve on an older host, using ZFS....
 
bhyveload is based on loader(8) and will present an interface identical
to the FreeBSD loader on the user's terminal. This behavior can be
changed by specifying a different OS loader.
Click to expand...
I am not sure it loads the kernel at this point.

I think the problem is because the host cannot 'translate' the ZFS features/module from the vm to the host's kernel.
Maybe using sysutils/openzfs-kmod will be able to do the 'translation'.
 
Phishfry said:
I agree. More of a stage 2 loader.
Click to expand...
That makes sense, but probably leaves the same problem, the host must know how to read whatever FS is used on the virtual disk...

Phishfry said:
I know the bhyve uefi-csm port was broken for a while.
Are you sure it is functional? Rebecca Cran has been doing most of the upkeep on the bhyve UEFI tiano stuff.
Click to expand...
Uhm, no ? I only have uefi (without csm) for my other virtual machines...
 
create another small disk image with an efi partitition and use that too boot the vm
or if you are brave extend the zvol, create an efi partition, change from mbr to gpt... :)
 
covacat said:
create another small disk image with an efi partitition and use that too boot the vm
or if you are brave extend the zvol, create an efi partition, change from mbr to gpt... :)
Click to expand...
Sure, if it's really uefi-csm being broken in general, that would be the way to solve the issue. Or I just don't upgrade that pool and keep using bhyveload(8) ;)
 
One thing I still wonder: Is this a supported and robust usage of bhyveload(8) (using a "user-space loader" from a newer version)?

Looking at this:
Code: 
➜ file /boot/userboot.so 
/boot/userboot.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, not stripped
I wonder why it's "dynamically linked"... but then:

Code: 
➜ ldd /boot/userboot.so 
/boot/userboot.so:

➜

Well, at least it does work for now, which is great ;)
 
You must log in or register to reply here.
Back
Top