UEFI ZFS boot: EFI on one disk, zroot on the other

I'm trying to set up UEFI boot to a ZFS filesystem, my setup is as follows:

# gpart show
=> 40 1875384928 nda0 GPT (894G)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 4194304 3 freebsd-swap (2.0G)
4728832 1870655488 4 freebsd-zfs (892G)
1875384320 648 - free - (324K)

=> 40 31251759024 da1 GPT (15T)
40 31251759024 1 freebsd-zfs (15T)

=> 34 31251759037 da0 GPT (15T)
34 6 - free - (3.0K)
40 31251759024 1 freebsd-zfs (15T)
31251759064 7 - free - (3.5K)

# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
zroot 14.5T 8.84T 5.70T - - 9% 60% 1.00x ONLINE -
zssd 888G 2.57M 888G - - 0% 0% 1.00x ONLINE -



The EFI partition lives on the SSD, whereas zroot lives on the two hard drives.

When I boot, the loader can't find the zroot.

# zpool get bootfs
NAME PROPERTY VALUE SOURCE
zroot bootfs zroot/ROOT/default local
zssd bootfs - default


However, if I manually...

set currdev=zfs:/zroot/ROOT/default:
load /boot/kernel/kernel
load /boot/kernel/zfs.ko
load /boot/kernel/mpr.ko
boot


Success! It boots just fine.
I noticed the environment variables in the loader are full of references to zssd, but I can't figure out why it looks there and not zroot, or if and where those configurations are being sourced from.

For now, manually keying in the data works for me, but there has to be a better way.
I have placed loader.rc and loader.conf in the root of the EFI directory, next to the bootx64.efi, but it doesn't seem to care.
Any ideas?
 
When I boot, the loader can't find the zroot.

I noticed the environment variables in the loader are full of references to zssd, but I can't figure out why it looks there and not zroot,
The first file system located on the same disk as loader.efi(8) found will be started first. The loader requires additional instructions to boot the "zroot" pool or any other file system, which may be located on a different hard disk or on the same hard disk.

I have placed loader.rc and loader.conf in the root of the EFI directory, next to the bootx64.efi, but it doesn't seem to care.
They are ignored, the loader does not support those files from the ESP. A limited number of simple variables can be set in /efi/freebsd/loader.env at most, see loader.efi(8) and loader_simp(8).

Any ideas?
You have these options:
  1. Create UEFI menu entries, specifying the ZFS pools, choose boot pool from the UEFI device boot menu
  2. Specify "next boot" ZFS pool from the current running system (probably not what you want necessarily )
1.
Code:
# efibootmgr  -c  -a  -L  FBSDzssd  -l  nda0p1:/efi/freebsd/loader.efi  -e  "rootdev=zfs:zssd/ROOT/default:"

# efibootmgr  -c  -a  -L  FBSDzroot  -l  nda0p1:/efi/freebsd/loader.efi  -e  "rootdev=zfs:zroot/ROOT/default:"


2.
Code:
## set next boot pool "zssd"
# zfsbootcfg "zfs:zssd/ROOT/default:"

## set next boot pool "zroot"
# zfsbootcfg "zfs:zroot/ROOT/default:"

## show next boot pool
# zfsbootcfg

## clear the boot options
# zfsbootcfg ""

You could create shell aliases to type less
Code:
alias        bzssd='zfsbootcfg "zfs:zssd/ROOT/default:"'
alias        bzroot='zfsbootcfg "zfs:zroot/ROOT/default:"'
 
/efi/freebsd/loader.env has the disadvantage of being static. With a specific ZFS pool selected (rootdev variable), it will boot always that pool. To boot from another ZFS pool, the user must intervene manually (setting currdevat the loader prompt, for example).

On the other hand, UEFI menu entries, each with a separate boot pool specified, can be chosen dynamically.

Not sure why, but it's not (at least well) documented officially.

Maybe you missed it but it's documented quit well in loader.efi(8) manual, even with an example:
Rich (BB code):
   Additional Environment Variables
     loader.efi loads some extra variables early in startup from
     /efi/freebsd/loader.env from the EFI partition.  Only simple variables
     can be set here.  It can be useful to specify the root filesystem:

           rootdev=disk0s1a
loader_simp(8) provides the syntax for currdev, rootdev of ZFS file systems:
Rich (BB code):
      currdev   Selects the default device to loader the kernel from.  The
               syntax is:
                     loader_device:
               or
                     zfs:dataset:
               Examples:
                     disk0p2:
                     zfs:zroot/ROOT/default:

      rootdev   By default the value of currdev is used to set the root file
               system when the kernel is booted.  This can be overridden by
               setting rootdev explicitly.

The user can always grep through all the manual pages instead of digging around in them to find the appropriate manual (using here textproc/ugrep, it prints a nice colored list of pattern occurrences, and it's fast, userland grep(1) doesn't support "decompress" option (to decompress *.gz manuals), and zgrep(1) doesn't support "recursive" option).
Code:
# ugrep  -rIzw  loader.env  /usr/share/man
/usr/share/man/man5/config.5.gz
   126: .Va loader_env.disabled=1
   143: .Va loader_env.disabled .

/usr/share/man/man8/loader.efi.8.gz
   211: .Pa /efi/freebsd/loader.env


# ugrep  -rIz  currdev  /usr/share/man
...
/usr/share/man/man8/loader_4th.8.gz
    71: .Va currdev
   405: .Va currdev
   444: .Va currdev
   505: set currdev=zfs:tank/ROOT/knowngood:

/usr/share/man/man8/loader_lua.8.gz
    68: .Va currdev
   146: .Va currdev
   185: .Va currdev
   230: set currdev=zfs:tank/ROOT/knowngood:

/usr/share/man/man8/loader_simp.8.gz
    67: .Va currdev
   430: .It Va currdev
   498: .Va currdev
   659: .Va currdev
   717: set currdev=zfs:tank/ROOT/knowngood:
 
/efi/freebsd/loader.env has the disadvantage of being static. With a specific ZFS pool selected (rootdev variable), it will boot always that pool. To boot from another ZFS pool, the user must intervene manually (setting currdevat the loader prompt, for example).

On the other hand, UEFI menu entries, each with a separate boot pool specified, can be chosen dynamically.



Maybe you missed it but it's documented quit well in loader.efi(8) manual, even with an example:
Rich (BB code):
   Additional Environment Variables
     loader.efi loads some extra variables early in startup from
     /efi/freebsd/loader.env from the EFI partition.  Only simple variables
     can be set here.  It can be useful to specify the root filesystem:

           rootdev=disk0s1a
loader_simp(8) provides the syntax for currdev, rootdev of ZFS file systems:
Rich (BB code):
      currdev   Selects the default device to loader the kernel from.  The
               syntax is:
                     loader_device:
               or
                     zfs:dataset:
               Examples:
                     disk0p2:
                     zfs:zroot/ROOT/default:

      rootdev   By default the value of currdev is used to set the root file
               system when the kernel is booted.  This can be overridden by
               setting rootdev explicitly.

The user can always grep through all the manual pages instead of digging around in them to find the appropriate manual (using here textproc/ugrep, it prints a nice colored list of pattern occurrences, and it's fast, userland grep(1) doesn't support "decompress" option (to decompress *.gz manuals), and zgrep(1) doesn't support "recursive" option).
Code:
# ugrep  -rIzw  loader.env  /usr/share/man
/usr/share/man/man5/config.5.gz
   126: .Va loader_env.disabled=1
   143: .Va loader_env.disabled .

/usr/share/man/man8/loader.efi.8.gz
   211: .Pa /efi/freebsd/loader.env


# ugrep  -rIz  currdev  /usr/share/man
...
/usr/share/man/man8/loader_4th.8.gz
    71: .Va currdev
   405: .Va currdev
   444: .Va currdev
   505: set currdev=zfs:tank/ROOT/knowngood:

/usr/share/man/man8/loader_lua.8.gz
    68: .Va currdev
   146: .Va currdev
   185: .Va currdev
   230: set currdev=zfs:tank/ROOT/knowngood:

/usr/share/man/man8/loader_simp.8.gz
    67: .Va currdev
   430: .It Va currdev
   498: .Va currdev
   659: .Va currdev
   717: set currdev=zfs:tank/ROOT/knowngood:
The problem would be loader.env does NOT have its manpage, even though loader.conf(5) has one.

And if static config is the issue, another way would be to use patched boot1.efi with PR 207940.
Why I've not introduced this at the first place is because boot1.efi is already deprecated (not deleted, though).
I hope loader.efi implement the functionality of the patch, with brushing up (i.e., showing device name expected to be assigned by kernel [GEOM] like ada0p2, nda1p5, ... which both of latest patch cannot).
 
Back
Top