bootloader issue on FreeBSD 15.0 -> 15.1 upgrade

dvl@ · 2026-07-17T21:11:41+0100

Hello,

I'm looking for someone with more bootloader knowledge and experience than I have. A plan appears at the end of this post.

I've been updating several FreeBSD 15.0 hosts to 15.1 using pkgbase. It concerns an EFI booting host.

The boot devices are not clearly shown here, as expected (based on https://www.freebsd.org/releases/15.1R/upgrading/#upgrade-loader-uefi)

Code:

[19:11 r730-01 root ~] # sysctl machdep.bootmethod
machdep.bootmethod: UEFI
[19:11 r730-01 root ~] # efibootmgr -v
BootCurrent: 0000
Timeout    : 0 seconds
BootOrder  : 0000, 0001
+Boot0000* EFI Fixed Disk Boot Device 1 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x4,0x0,0x0)/HD(1,GPT,6e0689af-ca20-11f0-9f70-ecf4bbefc954,0x800,0x64000)
 Boot0001  EFI Fixed Disk Boot Device 2 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x5,0x0,0x0)/HD(1,GPT,709f19af-ca20-11f0-9f70-ecf4bbefc954,0x800,0x64000)


Unreferenced Variables:
 Boot0004* Hard drive C: VenHw(d6c0639f-c705-4eb9-aa4f-5802d8823de6)
 Boot0005* IBA GE Slot 0100 v1588 BBS(0x80,IBA GE Slot 0100 v1588,0x0)

I also ran some magic mentioned by bsdimp (when I asked about another similar host issue):

Code:

[19:12 r730-01 root ~] # efivar cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath --utf8
cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
\EFI\BOOT\BOOTX64.EFI

[19:14 r730-01 root ~] # sudo efivar cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderDev --utf8
cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderDev
PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x4,0x0,0x0)/HD(1,GPT,6E0689AF-CA20-11F0-9F70-ECF4BBEFC954,0x800,0x64000)

That tells me it's booting from \EFI\BOOT\BOOTX64.EFI

I know ada0 and ada1 are my boot drives:

Code:

[19:19 r730-01 root ~] # zpool status zroot      
  pool: zroot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
    The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:33 with 0 errors on Thu Jul 16 04:02:31 2026
config:

    NAME                               STATE     READ WRITE CKSUM
    zroot                              ONLINE       0     0     0
      mirror-0                         ONLINE       0     0     0
        gpt/zfs0_20170718AA0000185556  ONLINE       0     0     0
        gpt/zfs1_20170719AA1178164201  ONLINE       0     0     0

errors: No known data errors


=>       40  242255584  ada0  GPT  (116G)
         40       2008        - free -  (1004K)
       2048     409600     1  efi0_20170718AA0000185556  (200M)
     411648   16777216     2  swap0  (8.0G)
   17188864  225066760     3  zfs0_20170718AA0000185556  (107G)

=>       40  242255584  ada1  GPT  (116G)
         40       2008        - free -  (1004K)
       2048     409600     1  efi1_20170719AA1178164201  (200M)
     411648   16777216     2  swap1  (8.0G)
   17188864  225066760     3  zfs1_20170719AA1178164201  (107G)

With that in mind, I did this mount and looked around:

Code:

[19:18 r730-01 root ~] # mount_msdosfs /dev/ada0p1 /boot/efi
[19:19 r730-01 root ~] # ls -l /boot/efi
total 16
drwxr-xr-x  1 root wheel 8192 Dec  8  2025 EFI
[19:19 r730-01 root ~] # find /boot/efi
/boot/efi
/boot/efi/EFI
/boot/efi/EFI/BOOT
/boot/efi/EFI/BOOT/BOOTX64.EFI

That is consistent with what the efivar command output above.

With that in mind, my plan is to do the following:

cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
umount /boot/efi
mount_msdosfs /dev/ada1p1 /boot/efi
cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
umount /boot/efi
shutdown -r now
profit

Does this seem reasonable?

I agree, the second umount[CMD] is not necessary.

mer · 2026-07-17T21:38:47+0100

Are ada0 and ada1 a mirror pair? I'm assuming they are, with a second partition the ZFS mirror.
Not that ada0 and ada1 are a geom mirror and ada[01]p1 are not mirrored.

If those assumptions are correct then I agree with your "plan" steps to update the efi partititions on the individual boot drives. I mildly disagree with "step #5 the second umount being unnecessary" (If I manually mount it I want to manually umount it before a reboot just because OCD)

Phishfry · 2026-07-17T21:39:58+0100

dvl@ said:
With that in mind, my plan is to do the following:

cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI

umount /boot/efi

That is consistent with the handbook. I don't automatically mount EFI in fstab but I do comment it out.
But yesterday I end up just manually mounting my Mirror EFI partition and running command #1

I dunno about ZFS mirror but I just did upgraded my firewall with gmirror pair of ssd and my RAID is "On the Mirror".
So I mount -t msdosfs /dev/mirror/gm0p1 /mnt and cp over loader.efi.
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi

Handbook had so many paths it messed me up. I finally just searched the mount and found it.

loader.efi

man.freebsd.org

cmp /boot/efi/EFI/FREEBSD/LOADER.EFI /boot/efi/EFI/BOOT/BOOTX64.EFI
I misread this and was searching all over for/EFI/FREEBSD/

mer · 2026-07-17T21:48:11+0100

Phishfry said:
I dunno about ZFS mirror but I just did upgraded my firewall with gmirror pair of ssd and my RAID is "On the Mirror".
So I mount -t mosdosfs /dev/gm0p1 /mnt and cp over loader.efi.

Is gm0p1 a partition off the mirror? Not one of the devices in the mirror? If so then I think that's good.

I guess I'm asking (fake device names, and may not be correct):
system has ada0 and ada1. Do geom mirror ada0 ada1 so the "whole" device for ada0 and ada1 are mirrored. Lets say that mirror is named mirror1 so we get a /dev/mirror1 and I then do gpart /dev/mirror1 to create the efi partitions. Then if I mount /dev/mirror1p1 and copy to it the data goes to both devices because of GEOM.

I think Phishfry that is roughly what you are saying?

I guess "don't geom mirror devices, partition, then create zfs mirrors on the geom mirrored partitions"

Phishfry · 2026-07-17T21:52:48+0100

My Mirror gets written to the EFI when I do the installation.
I drop out of installer and setup gmirror and format and return to installation.
So when the installer writes the EFI bits it goes into the gmirror UUID not a single disk.

Code:

 # efibootmgr -v
Boot to FW : false
BootCurrent: 0007
Timeout    : 1 seconds
BootOrder  : 0007, 000A, 0004
+Boot0007* UEFI OS HD(1,GPT,51cc9663-1b06-11ec-89b9-00e0ed5d51ce,0x28,0x64000)/File(\EFI\BOOT\BOOTX64.EFI)
                      mirror/gm0p1:/EFI/BOOT/BOOTX64.EFI /boot/efi//EFI/BOOT/BOOTX64.EFI
 Boot000A* UEFI: FiD 2.5 SATA10000 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x1,0xffff,0x0)/HD(1,GPT,51cc9663-1b06-11ec-89b9-00e0ed5d51ce,0x28,0x64000)
                                  VenHw(2d6447ef-3bc9-41a0-ac19-4d51d01b4ce6,300032003000310036003000300033004100410030004200300030003000300030003000340031000000)
 Boot0004  UEFI: Built-in EFI Shell  VenMedia(5023b95c-db26-429b-a648-bd47664c8012)


Unreferenced Variables:

mer · 2026-07-17T21:56:49+0100

Phishfry said:

My Mirror gets written to the EFI when I do the installation.
I drop out of installer and setup gmirror and format and return to installation.
So when the installer writes the EFI bits it goes into the gmirror UUID not a single disk.

Code:

 # efibootmgr -v
Boot to FW : false
BootCurrent: 0007
Timeout    : 1 seconds
BootOrder  : 0007, 000A, 0004
+Boot0007* UEFI OS HD(1,GPT,51cc9663-1b06-11ec-89b9-00e0ed5d51ce,0x28,0x64000)/File(\EFI\BOOT\BOOTX64.EFI)
                      mirror/gm0p1:/EFI/BOOT/BOOTX64.EFI /boot/efi//EFI/BOOT/BOOTX64.EFI
 Boot000A* UEFI: FiD 2.5 SATA10000 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x1,0xffff,0x0)/HD(1,GPT,51cc9663-1b06-11ec-89b9-00e0ed5d51ce,0x28,0x64000)
                                  VenHw(2d6447ef-3bc9-41a0-ac19-4d51d01b4ce6,300032003000310036003000300033004100410030004200300030003000300030003000340031000000)
 Boot0004  UEFI: Built-in EFI Shell  VenMedia(5023b95c-db26-429b-a648-bd47664c8012)


Unreferenced Variables:

Thanks for clarification. Sound almost like something from a Michael Lucas book

(I have theoretical knowledge of doing that not practical)

Phishfry · 2026-07-17T22:03:26+0100

i must admit I do refer to my only How-To when I create new arrays.

Code:

gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ada0 (ACTIVE)
                      ada1 (ACTIVE)

Puny little Innodisk 16GB SSD. Very old. Smart does not work correctly.

But if I were Dan I would look at the BIOS and see what it is booting off of exactly.

Mine is set to "UEFI OS" which you can also see from efibootmgr.

Phishfry said:
Boot0007* UEFI OS HD

Phishfry · 2026-07-17T22:26:27+0100

+Boot0000* EFI Fixed Disk Boot Device 1 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x4,0x0,0x0)/HD(1,GPT,6e0689af-ca20-11f0-9f70-ecf4bbefc954,0x800,0x64000)

Boot0001 EFI Fixed Disk Boot Device 2 PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0x5,0x0,0x0)/HD(1,GPT,709f19af-ca20-11f0-9f70-ecf4bbefc954,0x800,0x64000)<br>

Looking at Dans output it does look like the BIOS only see's individual drives.
Like I said I am clueless about ZFS but I would have assumed a BIOS EFI setting for /zroot/mirror-0/

dvl@ · 2026-07-17T22:37:02+0100

mer said:
Are ada0 and ada1 a mirror pair? I'm assuming they are, with a second partition the ZFS mirror.
Not that ada0 and ada1 are a geom mirror and ada[01]p1 are not mirrored.

Yes, they are each part of the zroot mirror.

mer said:
If those assumptions are correct then I agree with your "plan" steps to update the efi partititions on the individual boot drives. I mildly disagree with "step #5 the second umount being unnecessary" (If I manually mount it I want to manually umount it before a reboot just because OCD)

Technically, I think the OS will umount them just fine, but yes, if you mount, umount.

dvl@ · 2026-07-17T22:44:42+0100

dvl@ said:
With that in mind, my plan is to do the following:

cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI

umount /boot/efi

mount_msdosfs /dev/ada1p1 /boot/efi

cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI

umount /boot/efi

shutdown -r now

profit

Does this seem reasonable?

I agree, the second umount[CMD] is not necessary.

Here goes:

Code:

[21:42 r730-01 root ~] # cmp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
/boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI differ: char 158, line 3
[21:42 r730-01 root ~] # cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI

[21:42 r730-01 root ~] # umount /boot/efi

[21:42 r730-01 root ~] # mount_msdosfs /dev/ada1p1 /boot/efi
[21:42 r730-01 root ~] # cmp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
/boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI differ: char 158, line 3
[21:43 r730-01 root ~] # cp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
[21:43 r730-01 root ~] # cmp /boot/loader.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
[21:43 r730-01 root ~] # umount /boot/efi

[21:43 r730-01 root ~] #

That proves both updates were required. They are no mirrors. They are separate entities.

reboot in progress.

Phishfry · 2026-07-17T22:50:17+0100

I was looking online for ZFS zroot UUID and the first thing that poped up was you.

Creating a mirror from your zroot – Dan Langille's Other Diary

ZFS does appear different than UFS in that the drives have a UUID but the mirror-0 does not. (From what I can tell)

dvl@ · 2026-07-17T22:52:01+0100

dvl@ said:
reboot in progress.

Success.

Code:

[17:46 pro02 dvl ~] % r730
Last login: Fri Jul 17 21:42:13 2026 from pro02.startpoint.vpn.unixathome.org
[21:50 r730-01 dvl ~] % freebsd-version -ukr
15.1-RELEASE-p1
15.1-RELEASE-p1
15.1-RELEASE-p1
[21:50 r730-01 dvl ~] %

Thank you for coming to my TED talk.

dvl@ · 2026-07-17T22:52:44+0100

Phishfry said:
I was looking online for ZFS zroot UUID and the first thing that poped up was you.

Creating a mirror from your zroot – Dan Langille's Other Diary

ZFS does appear different than UFS in that the drives have a UUID but the mirror-0 does not. (From what I can tell)

I chuckle when I go searching for something and find I've already encountered it before....

Phishfry · 2026-07-17T23:02:00+0100

OK cool.
Can we talk philosophical?

Shouldn't you really have a EFIBOOTMGR setting for zpool/mirror-0 ?
zpool get guid zroot That would get you the GUID to set into EFI.

You have two EFI partitions and I don't think you need to. Sure it works but it seems to irk me.
You are booting off a ZFS Pool or are you? Everybody should be in the pool. One EFI mirrored. Failure handled by ZFS pool.
Please anybody correct me if I am thinking wrongly.

Phishfry · 2026-07-17T23:34:21+0100

What I am saying is this:
When SATA1 fails will BIOS rollover automatically?
Lets say your at a Phillies Game and you are updating your server over SSH and you do a reboot.
Ruttro.
Disk has failed at BIOS and is stalled....
Got to leave the game to attend to your server....
But if you were booting off "UEFI OS" via /zpool/mirror-0/ then BIOS only needs a live pool. Either Drive.
So what I am getting at is you can create new EFIBOOTMGR settings and with GUID you can point to the ZFS Pool.
Letting ZFS do its job of data redundancy.
That is my theory. Is anybody doing that? You also have BE and I don't know how they act. A pool is a pool.

mer · 2026-07-18T00:02:13+0100

dvl@ said:
I chuckle when I go searching for something and find I've already encountered it before....

This is one of the most ironic "I chuckle..." in a long time

(I ran across your ZFS on partitions a long time ago and followed it because it makes sense)

So back up in #9 and #10 that implies to me "standard partitioning". Separate devices, gpart on each that creates a EFI partitition, a swap (maybe) and a zfs partition. Then the zfs partitions (vdevs) are used to create a zfs mirror. That means the 2 efi partitions are separate so updates need to happen on each.
That is different from what I think Phishfry is doing (create GEOM mirror of raw devices, then partition the mirror)

I know one of my systems 2 devices, partitioned identically, but not geom mirror. boot partition, swap partition, zfs partition. zfs partitions made up a zfs mirror and I wound up forgetting to update BOTH boot partitions once. Bios decided to try and boot off the one I did not update, it did good until it went "oh crap the zfs mirror was zpool upgrade and I can't handle that". Totally my fault but a good lesson.

drhowarddrfine · 2026-07-18T00:14:06+0100

dvl@ said:
I chuckle when I go searching for something and find I've already encountered it before....

A couple of times I found some great answers about programming on StackOverflow by Googling and afterwards realized they were mine from long ago.

Phishfry · 2026-07-18T00:16:21+0100

I call bullcrap on myself.

Phishfry said:
But if you were booting off "UEFI OS" via /zpool/mirror-0/ then BIOS only needs a live pool. Either Drive.

If BIOS stalls for bad disk the Mirror or SATA problem don't matter. POST check goes first. If POST stalls then you will be attending to the machine.

bakul · 2026-07-18T00:37:19+0100

I periodically copy /boot/loader.efi to /boot/efi/efi/{boot/bootx64.efi,freebsd/loader.efi} to avoid this sort of problem, especially on some machines where I follow 15.1-STABLE or -CURRENT.

On my 15.1 machine:

 $ md5 -r /boot/efi/efi/*/* /boot/loader.efi

3df7c6bbc56827cf6fab1da2091b31b1 /boot/efi/efi/boot/bootx64.efi

3df7c6bbc56827cf6fab1da2091b31b1 /boot/efi/efi/freebsd/loader.efi

3df7c6bbc56827cf6fab1da2091b31b1 /boot/loader.efi