Solved How do I share a ZFS share via NFSv4 over wireguard and a PF firewall?

[zilti]

I can't seem to figure this out. I got to the point where a mount would "work", ls shows nothing, but I can cd through the folder structure of the mounted NFS share. I have no idea what I am doing wrong, and neither does the Internet, apparently...

NFS Server / Wireguard Client config

/etc/rc.conf:

hostname="generic"
ifconfig_DEFAULT="DHCP inet6 accept_rtadv"
sshd_enable="YES"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
growfs_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
ntpd_enable="YES"
zfs_enable="YES"
nfs_server_enable="YES"
nfsv4_server_enable="YES"
wireguard_enable=YES
wireguard_interfaces="wg0"
nfsuserd_enable="YES"

/etc/wireguard/wg0.conf:

[Interface]
Address = 172.16.0.1/12
PrivateKey = REDACTED

[Peer]
PublicKey = REDACTED
AllowedIPs = 0.0.0.0/0
Endpoint = lyrion.ch:51820

/etc/exports:

V4: /

ZFS has been configured with zfs set sharenfs="maproot=root,172.16.0.0/12,rw,fsid=root,no_root_squash,no_subtree_check" raidstore.

NFS Client / Wireguard Server config

/etc/rc.conf:

clear_tmp_enable="YES"
sshd_enable="YES"
sendmail_enable="NONE"
qemu_guest_agent_enable="YES"
qemu_guest_agent_flags="-d -v -l /var/log/qemu-ga.log"
zfs_enable="YES"
hostname=REDACTED
ifconfig_vtnet0="DHCP"
ntpd_enable="YES"
ipv6_activate_all_interfaces="YES"
ifconfig_vtnet0_ipv6="inet6 DHCP accept_rtadv"
rtsold_enable="YES"
dhclient_program="/usr/local/sbin/dual-dhclient"
dhclient_enable="YES"
blacklistd_enable="NO"
pf_enable="YES"
cloned_interfaces="lo1"
ifconfig_lo1_name="bastille0"
ifconfig_bastille0="inet 10.0.0.1 netmask 255.0.0.0"
bastille_enable="NO"
bastille_rcorder="YES"
nginx_enable="NO"
haproxy_enable="YES"

wireguard_enable="YES"
wireguard_interfaces="wg0"
gateway_enable="NO"
nfs_client_enable="YES"
nfsuserd_enable="YES"
nfs_client_flags="-n 4"

/etc/wireguard/wg0.conf:

[Interface]
Address = 172.16.0.2/12
ListenPort = 51820
PrivateKey = REDACTED

[Peer]
PublicKey = REDACTED
AllowedIPs = 172.16.0.1/12

/etc/pf.conf:

ext_if="vtnet0"

set skip on lo0
set skip on bastille0
set block-policy return
scrub in on $ext_if all fragment reassemble

wg_if="wg0"
nat on $ext_if from $wg_if:network to any -> ($ext_if:0)

table <jails> persist
nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"

block in all
pass out all
antispoof for $ext_if inet
pass in inet proto tcp from any to any port ssh flags S/SA modulate state
pass in inet proto tcp from any to any port 443 flags S/SA modulate state
pass in inet proto udp from any to any port 51820

pass proto ipv6-icmp from any to any
pass quick on $wg_if

And now I'd like to mount the ZFS filesystem raidstore/mailserver. My attempt so far was mount -t nfs -o nfsv4 -o rw 172.16.0.1:raidstore/mailserver /mnt/mailserver, which gave me no errors, but while I can navigate the file system, I can neither see anything with ls, nor write to it - trying to write gives me an unhelpful "Input/Output Error". What am I missing?

 

[T-Daemon]

ZFS has been configured with zfs set sharenfs="maproot=root,172.16.0.0/12,rw,fsid=root,no_root_squash,no_subtree_check" raidstore.

Please remove rw,fsid=root,no_root_squash,no_subtree_check. Those are invalid FreeBSD exports(5) settings.

Set:

zfs set sharenfs="maproot=root,network=172.16.0.0/12"

Check also /var/log/messages for mountd(8) error messages.

ZFS exports(5) settings are stored in /etc/zfs/exports.

 

[zilti]

Thanks, I added those options as a "last effort" before asking here - originally I of course tried the defaults by just setting it to on, and then I had maproot=root,172.16.0.0/12 - so, not quite what you suggested. This does not solve the problem though, unfortunately - it's still just as it was before.

Is /etc/zfs/exports supposed to be auto-populated by zfs, or is that for me to set defaults? Because it is empty on my system.

There's nothing logged to /var/log/messages when I use mount -t nfs -o nfsv4 -o rw 172.16.0.1:raidstore/mailserver /mnt/mailserver, but when I add a leading slash to the remote location like this: 172.16.0.1:/raidstore/mailserver, then I get "kernel: nfsv4 no file handle: usually means the file system is not exported on the NFSv4 server".

 

[zilti]

Ah! I called zfs share the wrong way - my zpool is encrypted, so while zfs share -a seemingly works, it does nothing. What I needed to do was zfs share -l -a!