ZFS zfs encryption and mount reporting question

[lixen]

I see a zfs behavior that I can't understand and hope someone can clarify it for me.

System overview:
- FreeBSD 14.2 host, nda1 geli encrypted /zroot/
- Archlinux guest in Bhyve, disk is in the /zroot/bhyve/
- Shared zfs partition at nda2p1, zfs encrypted, accessible to FreeBSD and via nfs to Arch (mounted at /mnt/work/
- nda0 is windows boot disk, unrelated.

Issue:
- On system boot, I am prompted only to enter geli passphrase (different from shared zfs passphrase).
- zfs is set to prompt for password on /zshare/work dataset. However, it never asks me for the passphrase.
- zshare/work is available on FreeBSD and Arch (all R/W and permissions work fine), but zfs reports it's not decrypted (can be some wrong error type) and shows that it's not mounted.

Question:
- Am I missing some configuration or do I not correctly understand how it should work? Appreciate the help.

Disk setup:

DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
nda0 0:108 1.9T GPT - -
<FREE> -:- 1.0M - - -
nda0p1 0:110 500M efi gpt/EFI -
nda0p2 0:112 128M ms-reserved gpt/MS/reserved -
nda0p3 0:114 1.9T ms-basic-data gpt/MS/data/basic -
nda0p4 0:116 999M ms-recovery ntfs/Recovery -
<FREE> -:- 328K - - -
nda1 0:118 1.8T GPT - -
nda1p1 0:138 260M efi gpt/efiboot0 /boot/efi
<FREE> -:- 1.0M - - -
nda1p2 0:140 96G freebsd-swap gpt/freebsd-swap SWAP
nda1p2.eli 0:229 96G freebsd-swap - SWAP
nda1p3 0:142 1.7T freebsd-zfs gpt/freebsd-zfs <ZFS>
nda1p3.eli 0:148 1.7T zfs - -
<FREE> -:- 68K - - -
nda2 0:120 1.8T GPT - -
nda2p1 0:144 500G freebsd-zfs gpt/freebsd-zfs <ZFS>
<FREE> -:- 1.3T - - -

Other configuration and relevant info (the red lines are the point of my question that don't seem to match the reality of the situation):

❯ zfs get encryption,keyformat,keylocation,mounted,mountpoint zshare/work
NAME PROPERTY VALUE SOURCE
zshare/work encryption aes-256-gcm -
zshare/work keyformat passphrase -
zshare/work keylocation prompt local
zshare/work mounted no -
zshare/work mountpoint /zshare/work default

❯ zfs get keystatus zshare/work
NAME PROPERTY VALUE SOURCE
zshare/work keystatus unavailable -

❯ mount | grep /zshare/work

❯ zfs get all zshare
NAME PROPERTY VALUE SOURCE
zshare type filesystem -
zshare creation Tue May 13 19:12 2025 -
zshare used 940K -
zshare available 481G -
zshare referenced 104K -
zshare compressratio 1.00x -
zshare mounted yes -
zshare quota none default
zshare reservation none default
zshare recordsize 128K default
zshare mountpoint /zshare default
zshare sharenfs off default
zshare checksum on default
zshare compression lz4 local
zshare atime off local
zshare devices on default
zshare exec on default
zshare setuid on default
zshare readonly off default
zshare jailed off default
zshare snapdir hidden default
zshare aclmode discard default
zshare aclinherit restricted default
zshare createtxg 1 -
zshare canmount on default
zshare xattr on default
zshare copies 1 default
zshare version 5 -
zshare utf8only off -
zshare normalization none -
zshare casesensitivity sensitive -
zshare vscan off default
zshare nbmand off default
zshare sharesmb off default
zshare refquota none default
zshare refreservation none default
zshare guid 5072130938392396252 -
zshare primarycache all default
zshare secondarycache all default
zshare usedbysnapshots 0B -
zshare usedbydataset 104K -
zshare usedbychildren 836K -
zshare usedbyrefreservation 0B -
zshare logbias latency default
zshare objsetid 54 -
zshare dedup off default
zshare mlslabel none default
zshare sync standard default
zshare dnodesize legacy default
zshare refcompressratio 1.00x -
zshare written 104K -
zshare logicalused 316K -
zshare logicalreferenced 46.5K -
zshare volmode default default
zshare filesystem_limit none default
zshare snapshot_limit none default
zshare filesystem_count none default
zshare snapshot_count none default
zshare snapdev hidden default
zshare acltype nfsv4 default
zshare context none default
zshare fscontext none default
zshare defcontext none default
zshare rootcontext none default
zshare relatime on default
zshare redundant_metadata all default
zshare overlay on default
zshare encryption off default
zshare keylocation none default
zshare keyformat none default
zshare pbkdf2iters 0 default
zshare special_small_blocks 0 default
zshare prefetch all default

❯ zfs get all zshare/work
NAME PROPERTY VALUE SOURCE
zshare/work type filesystem -
zshare/work creation Wed May 14 4:44 2025 -
zshare/work used 200K -
zshare/work available 481G -
zshare/work referenced 200K -
zshare/work compressratio 1.00x -
zshare/work mounted no -
zshare/work quota none default
zshare/work reservation none default
zshare/work recordsize 128K default
zshare/work mountpoint /zshare/work default
zshare/work sharenfs off default
zshare/work checksum on default
zshare/work compression lz4 inherited from zshare
zshare/work atime off inherited from zshare
zshare/work devices on default
zshare/work exec on default
zshare/work setuid on default
zshare/work readonly off default
zshare/work jailed off default
zshare/work snapdir hidden default
zshare/work aclmode discard default
zshare/work aclinherit restricted default
zshare/work createtxg 6796 -
zshare/work canmount on default
zshare/work xattr on default
zshare/work copies 1 default
zshare/work version 5 -
zshare/work utf8only off -
zshare/work normalization none -
zshare/work casesensitivity sensitive -
zshare/work vscan off default
zshare/work nbmand off default
zshare/work sharesmb off default
zshare/work refquota none default
zshare/work refreservation none default
zshare/work guid 16843295700568857199 -
zshare/work primarycache all default
zshare/work secondarycache all default
zshare/work usedbysnapshots 0B -
zshare/work usedbydataset 200K -
zshare/work usedbychildren 0B -
zshare/work usedbyrefreservation 0B -
zshare/work logbias latency default
zshare/work objsetid 68 -
zshare/work dedup off default
zshare/work mlslabel none default
zshare/work sync standard default
zshare/work dnodesize legacy default
zshare/work refcompressratio 1.00x -
zshare/work written 200K -
zshare/work logicalused 70K -
zshare/work logicalreferenced 70K -
zshare/work volmode default default
zshare/work filesystem_limit none default
zshare/work snapshot_limit none default
zshare/work filesystem_count none default
zshare/work snapshot_count none default
zshare/work snapdev hidden default
zshare/work acltype nfsv4 default
zshare/work context none default
zshare/work fscontext none default
zshare/work defcontext none default
zshare/work rootcontext none default
zshare/work relatime on default
zshare/work redundant_metadata all default
zshare/work overlay on default
zshare/work encryption aes-256-gcm -
zshare/work keylocation prompt local
zshare/work keyformat passphrase -
zshare/work pbkdf2iters 350000 -
zshare/work encryptionroot zshare/work -
zshare/work keystatus unavailable -
zshare/work special_small_blocks 0 default
zshare/work prefetch all default

❯ geli list
Geom name: nda1p3.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: accelerated software
Version: 7
UsedKey: 0
Flags: BOOT, GELIBOOT, AUTORESIZE
KeysAllocated: 442
KeysTotal: 442
Providers:
1. Name: nda1p3.eli
Mediasize: 1897045946368 (1.7T)
Sectorsize: 4096
Mode: r1w1e1
Consumers:
1. Name: nda1p3
Mediasize: 1897045950464 (1.7T)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 103352893440
Mode: r1w1e1

Geom name: nda1p2.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 128
Crypto: accelerated software
Version: 7
Flags: ONETIME, W-DETACH, W-OPEN, AUTORESIZE
KeysAllocated: 24
KeysTotal: 24
Providers:
1. Name: nda1p2.eli
Mediasize: 103079215104 (96G)
Sectorsize: 4096
Mode: r1w1e0
Consumers:
1. Name: nda1p2
Mediasize: 103079215104 (96G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 273678336
Mode: r1w1e1

❯ cat /zroot/bhyve/arch/arch.conf
loader="uefi"

cpu=8
memory=24G
wired_memory="yes"

# Network Configuration
network0_type="virtio-net"
network0_switch="private"

# Storage Configuration
disk0_type="virtio-blk"
disk0_name="disk0.img"
disk0_size="500G"

# Graphics Configuration (for VNC access to the graphical installer and desktop)
graphics="yes"
graphics_listen="127.0.0.1"
graphics_res="1920x1080"
xhci_mouse="yes"

# Optional: Description
desc="Arch Linux VM"

# Other settings:
debug="yes"

 

[lixen]

I think I found the cause:
I had created a directory /zshare/work/ (unencrypted) in addition to the /zshare/work/ dataset (encrypted).
It was causing confusion and what I was seeing from the guest vm was the unencryped directory (I only had test files with the same name in the director/dataset, or I would've noticed earlier, access timestamp gave the clue).
nfs was also serving the unencrypted directory and was failing to serve the encrypted dataset.
After removing the problematic directory zfs behavior is now ok and nfs properly serves encrypted dataset if mounted on host.