outerbase: install script for remote-unlockable geli-encrypted root-on-zfs



Reaction score: 49
Messages: 85

Many people have wondered how to have a system with full encryption-at-rest, and be able to unlock it remotely on boot, without being physically next to the machine.

You can't really have both: If the system does an unattended boot far enough to accept SSH connections, a lot of it needs to be unencrypted (bootloader and kernel, some of userland, some home data like ~/.ssh/authorized_keys). On the other hand, if the system is fully encrypted, a passphrase needs to be entered by keyboard before the system can get going at all.

Common solutions (apart from virtualization) include serial consoles, remote management solutions like IPMI, or KVM-over-IP hardware. This is my idea of a solution, based on this and that previous work. It's an imperfect compromise—but I like it.

                                                             |    gpt/inner: GELI        |
                                                             |  +----------------------+ |
                           +----------------+  3) unlock     |  | gpt/inner.eli: zroot | |
                           | gpt/outer: ufs |--------------> |  |                      | |
+--------------+  1) boot  |                |----------------+->|     "inner base"     | |
| gpt/efi: ESP |---------->|  "outer base"  |  4) reboot -r  |  +----------------------+ |
+--------------+           +----------------+                +---------------------------+
                              2) ssh +

An install script to set up a system in this way is published here: freebsd-outerbase on GitHub. The github README is exhaustive, so here's just some of the highlights and drawbacks:

The Good
  • The unencrypted outer base is a squeaky-clean base system, containing nothing but a public SSH key for login and the unlock script.
  • The inner base is a fully functional root-on-zfs system atop a geli-encrypted partition.
  • reboot -r is a slick operation, with low risk of the hardware, BIOS or other circumstances messing with the reboot into the inner base (and potentially causing a hang that needs to be resolved physically).
  • The script supports installing a custom base.txz for the outer base. A src.conf is provided for such a minimal outer base system (244M installed + kernel).
  • partial Boot Environments support for the inner base (excluding /boot)
  • optional encrypted swap partition
The Bad
  • Both kernel and bootloader need to be shared between outer and inner base, and are therefore unencrypted.
  • Encryption-at-rest doesn't protect the system while it's unlocked and the keys are in memory.
    • For data that only needs to be unlocked part of the time, some encryption atop/beside the inner base could be used.
  • For the moment, the script only supports single-OS UEFI boot on amd64.
The Ugly
  • In the inner base system, /boot does not reside on zfs. It's a sunlink-protected symlink to the outer base (mounted at /outer/boot).
  • SSH host keys are either identical or separate between the inner and outer base.
    • If they're identical, it's a security concern for the private host keys to be sitting unencrypted with the outer base.
    • If they're separate, some arm-twisting is needed to convince clients to connect even though the same host presents two sets of server keys.
Hope this proves useful to someone :)