Upgrade a FreeBSD 10.3 Installation with ZFS on Root and Full Disk Encryption to 11.0


Original document.

Thanks to Allan Jude for steering me through this on IRC and Warren Block for his feedback.

The new boot loader in 11.0 is able to boot encrypted ZFS pools directly. Yes, that means you can have full disk encryption (FDE) with ZFS on root and boot environments (BEs)! However, after you upgrade from 10.3, some tinkering is necessary to get this working. The instructions that follow are for a ZFS mirror installation. The two disks (ada0 and ada1) each have the same partition layout: p1: freebsd-boot, p2: freebsd-zfs (boot pool), p3: swap, p4: freebsd-zfs (main pool). Specify your disk(s) and partition indices according to your setup.

Reencrypt the master key with only a passphrase. You can use the same passphrase as before.
geli setkey -k /boot/encryption.key ada0p4
geli setkey -k /boot/encryption.key ada1p4
Set the geliboot flag.
geli configure -g ada0p4
geli configure -g ada1p4
Remove the /boot symbolic link pointing to /bootpool/boot and copy /boot from /bootpool/ to /.
rm /boot
cp -r /bootpool/boot /
Install the GPT boot code into the boot partition.
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
Set the partition type of the old boot pool partition to freebsd-vinum, so it does not get detected by the boot code as a ZFS partition.
gpart modify -t freebsd-vinum -i 2 ada0
gpart modify -t freebsd-vinum -i 2 ada1
Remove geli_ada0p4_*, geom_eli_passphrase_prompt, and (optional) zpool_cache_* from /boot/loader.conf.

Set canmount=noauto for all BEs, including the default.
zfs set canmount=noauto zroot/ROOT/default
Reboot to confirm everything is working. If you are satisfied, you can destroy the old boot pool.
zpool destroy bootpool
Delete the old boot pool partitions.
gpart delete -i2 ada0
gpart delete -i2 ada1
Delete the old swap partitions.
swapoff -a
gpart delete -i3 ada0
gpart delete -i3 ada1
Use the reclaimed space for larger swap partitions.
gpart add -t freebsd-swap -l swap0 ada0
gpart add -t freebsd-swap -l swap1 ada1
Update /etc/fstab to use the new swap partition indices.
# Device          Mountpoint  FStype  Options  Dump  Pass#
/dev/ada0p2.eli   none        swap    sw       0     0
/dev/ada1p2.eli   none        swap    sw       0     0
Turn swap back on.
swapon -a