Other Strange Behavior with a geli encrypted root-on-ZFS partition, using keyfiles in an unencrypted removable boot partition.

Title lays out the basic issues. It seems documentation is vague and most sources/posts discussing this seem to be outdated to some degree.

I used the auto root on zfs option from the installer, and recently created an unecrypted UFS boot usb. This is meant to contain keyfiles that should decrypt the root pool.

My issues however is this. While after I "boot to multi user", i do see that it appears to have found the location of my geli key, but once I'm booting I'm prompted for the password inspite of having configured loader.conf and the geli volume to not do so.

Strangely, im prompted for the password for ada0p3, but the passphrase i added (simply to be able to boot because it doesnt appear to be using the keyfile that it has indeed detected, and without an accessible key i cannot use the system) is indicated to be false.
It then asks for the passphrase for the same volume, this time calling it gpt/zfs0. It then successfully decrypts the volume and boots successfully.

My loader.conf options appear accurate to the geli(8) documentation, the geli volume has the -b option to make it bootable, and was deprived of the -g option to prevent it asking for a password at boot.

The boot usb successfully mounts after booting per fstab configureation.

Im attaching all the useful outputs i can think of. I cannot for the life of me determine why it doesnt appear to use the keyfile, or why it tries to access the root pool as 2 different names, and why only 1 of them works?

Code:
===========================
===========================
$lsblk
DEVICE         MAJ:MIN SIZE TYPE                              LABEL MOUNT
ada0             0:110 224G GPT                                   - -
  ada0p1         0:111 512K freebsd-boot               gpt/gptboot0 -
  <FREE>         -:-   492K -                                     - -
  ada0p2         0:112  17G freebsd-swap                  gpt/swap0 SWAP
  ada0p2.eli     0:100  17G freebsd-swap                          - SWAP
  ada0p3         0:113 207G freebsd-zfs                    gpt/zfs0 <ZFS>
  <FREE>         -:-   580K -                                     - -
da0              0:145  15G GPT                                   - -
  da0p1          0:148 512K freebsd-boot               gpt/gptboot0 -
  <FREE>         -:-   492K -                                     - -
  da0p2          0:149 1.0G freebsd-ufs                    gpt/boot /usbboot
  <FREE>         -:-    14G -                                     - -
da1              0:146    - -                                     - -

===========================
===========================
$ cat /boot/loader.conf
vfs.root_mount_always_wait="1"
kern.goem.eli.debug="3"

aesni_load="YES"
geom_eli_load="YES"
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
cryptodev_load="YES"
zfs_load="YES"

geli_ada0p3_keyfile0_load="YES"
geli_ada0p3_keyfile0_type="ada0p3:geli_keyfile0"
geli_ada0p3_keyfile0_name="/boot/my.key"

kern.vty=vt
fuse_load="YES"

vfs.root.mountfrom="zfs:zroot/ROOT/default"

=============================
=============================

$ doas geli list                                                                      1 ⨯
Geom name: gpt/zfs0.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: accelerated software
Version: 7
UsedKey: 1
Flags: BOOT, AUTORESIZE
KeysAllocated: 52
KeysTotal: 52
Providers:
1. Name: gpt/zfs0.eli
   Mediasize: 221802131456 (207G)
   Sectorsize: 4096
   Mode: r1w1e1
Consumers:
1. Name: gpt/zfs0
   Mediasize: 221802135552 (207G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 18254659584
   Mode: r1w1e1

============================
============================

doas geli dump -v gpt/zfs0                                                          1 ⨯
Metadata on gpt/zfs0:
     magic: GEOM::ELI
   version: 7
     flags: 0x202
     ealgo: AES-XTS
    keylen: 256
  provsize: 221802135552
sectorsize: 4096
      keys: 0x03

.................

$ doas geli dump -v ada0p3                                                            1 ⨯
Metadata on ada0p3:
     magic: GEOM::ELI
   version: 7
     flags: 0x202
     ealgo: AES-XTS
    keylen: 256
  provsize: 221802135552
sectorsize: 4096
      keys: 0x03
iterations: 1880379


===========================
===========================

$cat /var/log/messages

.........................
GEOM_ELI: Wrong key for ada0p3. Tries left: 2.
May 14 11:57:42 env0 kernel: Enter passphrase for ada0p3: GEOM_ELI: Wrong key for ada0p3. Tries left: 1.
May 14 11:57:42 env0 kernel: Enter passphrase for ada0p3: GEOM_ELI: Wrong key for ada0p3. No tries left.
May 14 11:57:42 env0 kernel: GEOM_ELI: Device gpt/zfs0.eli created.
May 14 11:57:42 env0 kernel: GEOM_ELI: Encryption: AES-XTS 256
May 14 11:57:42 env0 kernel: GEOM_ELI:     Crypto: accelerated software
May 14 11:57:42 env0 kernel: GEOM_ELI: Device ada0p2.eli created.
May 14 11:57:42 env0 kernel: GEOM_ELI: Encryption: AES-XTS 128
May 14 11:57:42 env0 kernel: GEOM_ELI:     Crypto: accelerated software
:
 
Back
Top