Loader not found after upgrading the BIOS Boot Loader (Root on ZFS, freebsd-boot on USB)

I have two systems running for many years now. Upgrading the boot loader (via gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0) when upgrading from 14.3-RELEASE to 14.4-RELEASE (and even tested 15.1-RELEASE on one machine) left me with a system that does not boot without intervention. The loader on the root pool isn't found and I have to manually type zfs:zroot/ROOT/14.4-RELEASE:/boot/loader to get to the loader and boot the system.
If I remember it correctly I had the same issue on one of the earlier major upgrades and therefore left the old boot loader in place and stopped to upgrade the zfs pool. Sadly I now already upgraded the root pool on one of my machines.

Some more about my setup:
Code:
$ sysctl machdep.bootmethod
machdep.bootmethod: BIOS
$ zpool list -o name,bootfs
NAME   BOOTFS
zroot  zroot/ROOT/14.4-RELEASE
$ gpart show
=>        40  3907029088  ada0  GPT  (2T)
          40  3907029088     1  freebsd-zfs  (2T)

=>        40  3907029088  ada1  GPT  (2T)
          40  3907029088     1  freebsd-zfs  (2T)

=>        40  3907029088  ada4  GPT  (2T)
          40  3907029088     1  freebsd-zfs  (2T)

=>        40  3907029088  ada2  GPT  (2T)
          40  3907029088     1  freebsd-zfs  (2T)

=>        40  3907029088  ada3  GPT  (2T)
          40  3907029088     1  freebsd-zfs  (2T)

=>     40  7679920  da0  GPT  (4G)
       40       16       - free -  (8K)
       56     1024    1  freebsd-boot  (512K)
     1080  7678880       - free -  (4G)

Maybe anybody can see what I did wrong? Already tried to install the bootcode from the running os or a usb live system without success. I even upgraded one machine to 15.1 and used the 15.1 pmbr und gptzfsboot.
 
Just to verify:
Your boot device is da0
ada[0-4] is a different type of redundant vdev?
da0 is a boot device listed in BIOS?
 
Boot is on da0. ada[0-4]p1 are part of the zfs pool zroot.

For the exact messages before the boot prompt I am going to film the screen and copy them - but that might take a few days before I've got the time for that.
 
If it worked with a previous version of gptzfsboot, it sounds like a regression in its code.

Facts are BIOS booting is an ancient technology and will be less and less tested, furthermore like you use it, with an USB key; you might be the only one or so.

If you don't provide all the informations and tests, it's likely no FreeBSD dev will spend time at your possible future Problem Report.

This rise a question: can you try with UEFI? Because if your machine is UEFI capable, there are some chances you can workaround the problem like this.
 
What boot looks like:
Code:
BIOS drive C: is disk0
BIOS drive D: is disk1
BIOS drive E: is disk2
BIOS drive F: is disk3
BIOS drive G: is disk4
BIOS drive H: is disk5
BIOS drive I: is disk6
\
Can't find /boot/zfsloader

Can't find /boot/loader

FreeBSD/x86 boot
Default: disk-1:/boot/kernel/kernel
boot:

status shows the pool with the correct bootfs
zfs:zroot/ROOT/14.4-RELEASE:/boot/loader gives the loader and then the os

Yes, BIOS booting is ancient. But one of the machines isn't UEFI capable. Yes, I tried UEFI on the other one. Removed the freebsd-boot slice, replaced it with a 200M efi. But this also failed:
Code:
Consoles: EFI Console
    Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk1p1:
FreeBSD/amd64 EFI loader, Revision 3.0

  Command line arguments: loader.efi
  EFI base: 0x735d6000
  EFI version: 2.40
  EFI Firmware: American Megatrends (rev 5.11)
  Console: efi (0x20000000)
  Load Path: \EFI\BOOT\BOOTX64.EFI
  Load Device: PciRoot(0x0)/Pci(0x1D,0x0)/USB(0x0,0x0)/USB(0x4,0x0)/HD(1,GPT,4C4BEE6F-FC0C- <... screen cut on video>
,0x38,0x64000)
  BootCurrent: 0009
  BootOrder: 0007 0008 0009[*] 0003 0006
  BootInfo Path: VenHW(.....)
Ignoring Boot0009: No Media Path
Trying ESP: PciRoot(0x0)/Pci(0x1D,0x0)/USB(0x0,0x0)/USB(0x4,0x0)/HD(1,GPT,4C4<...>
,0x38,0x64000)
Setting currdev to disk1pi:
Failed to find bootable partition
<last line cut on video>
 
Back
Top