HowTo: FreeBSD 7/8.x full disk encryption on ZFS

frijsdijk · #1 March 20th, 2010, 21:46

Warning: the following howto will wipe your disk and delete all data on it! Be safe. Make backups first. Or use VirtualBox to test it first!

I've found a nice howto on how to install FreeBSD with full disk encryption on ZFS. This could a good choice for laptops where security is important. The ZFS on top of ELI makes the filesystem resilient against crashes (power cut or whatever), and it gives you a range of nice options to use! No more fsck after a ungraceful shutdown. Snapshots. To name a few.

The basis steps are:

** From sysinstall, follow your basic FreeBSD installation scenario, making these slices:

Code:

ad0s1a / (2GB)
ad0s1b swap (2GB)
ad0s1d /space (the rest)

** Install a minimal FreeBSD, enough to get you going (it will have to fit on your 2GB /). After installation completes, eject CDROM/DVD, and reboot from hard disk.

** Once booted, free up your ad0s1b, and copy the current root partition (ad0s1a) on to it

Code:

# swapoff -a
# newfs /dev/ad0s1b
# mount /dev/ad0s1b /mnt
# cd /
# tar cf - --one-file-system * | tar xpf - -C /mnt

** Now edit your /mnt/etc/fstab, and set your ad0s1b entry so that it is your root:

Before:

Code:

/dev/ad0s1b		none		swap	sw				0	0

After:

Code:

/dev/ad0s1b		/		ufs	rw				1	1

** Comment out all other entries.

** Now reboot again. At the bootloader, press space, and boot from the b-slice by entering:

Code:

0:ad(0,b)/boot/loader

Your system should boot normally. When you login, type 'mount' to verify that you are now booted and running from ad0s1b in stead of ad0s1a.

Now we can start the ELI/ZFS configuration on the largest part of the disk (ad0s1d).

Code:

# geli init -b -v -s 4096 /dev/ad0s1d

(you will be asked to enter a passphrase twice)

The -b causes eli to ask for the passphrase upon boot, when the kernel module is loaded by /boot/loader.conf.

Code:

# geli attach ad0s1d

(enter same passphrase)

We should now have a crypted ad0s1d.eli available (try 'geli list'). From this disk we create a ZFS pool:

Code:

# zpool create tank ad0s1d.eli

** Now we create /boot on the ZFS pool, which be on ad0s1a:

Code:

# newfs /dev/ad0s1a
# mkdir /tank/bootdir
# mount /dev/ad0s1a /tank/bootdir
# cp -Rp /boot /tank/bootdir/
# ln -sf /tank/bootdir/boot /tank/boot

The last line is only needed to make 'installkernel' behave normally, but it might be broken. See "fixing /boot" later on to fix this.

** Edit /tank/boot/loader.conf to contain the following:

Code:

zfs_load="YES"
geom_eli_load="YES"
hint.kbdmux.0.disabled=1

(note that this file is actually on ad0s1a!)

** Create /tank/bootdir/etc/fstab so the kernel knows what fs to mount the root from:

Code:

# mkdir /tank/bootdir/etc
# vi /tank/bootdir/etc/fstab

It should contain:

Code:

tank            /           zfs     rw     0     0

** Now create your actual fstab (/tank/etc/fstab):

Code:

tank            /           zfs     rw     0     0
/dev/ad0s1a     /bootdir    ufs     rw     1     1

You can use the ad0s1b later on to create swap again, or even encrypted swap (better of course).

** Now tell ZFS to prevent from re-mounting tank as it will be used for the root filesystem:

Code:

zfs set mountpoint=legacy tank

You can ignore the busy error message.

That's it. Reboot, and your done.

Fixing /boot:

I haven't figured this out yet, but as said your kernel (/boot) comes from ufs:ad0s1a. This means that once your booted, /boot should actually point to ad0s1a somehow, or kernel updates will never work.

To do this, I did, once booted, assuming ad0s1a is mounted in /bootdir:

Code:

cd / 
rm -rf boot (you now remove the boot from the zfs pool that is never used anyway)
ln -s bootdir/boot .

Now you should be able to run freebsd-update and such, or even make world/kernel.

Just always check that /boot/loader.conf (so actually that it ufs:ad0s1a/boot/loader.conf) is there and that eli and zfs are loaded there, otherwise you will not be able to boot.

dbi · #2 March 21st, 2010, 02:25

Pros: Encryption is great for laptops and other devices that can easily be lost/stolen.

I have some questions about possible cons, though.

1. Could you, please, share some info on the performance impact of GELI under ZFS?
2. Wouldn't putting GELI layer under ZFS break the self-healing and data integrity features of ZFS?

graudeejs · #3 March 21st, 2010, 10:07

1) works fine for me on my Desktop PC
2) Haven't tested self healing

frijsdijk · #4 March 21st, 2010, 10:38

There will be a performance hit, for sure. But performance is not the main focus here, data security is. I have been running FreeBSD 7.2 on my laptop a while back, on which at that time swap, /tmp, /var and /home were ELI crypted, but I wasn't running ZFS on top of it but GJOURNAL. Only my /usr wasn't crypted. But I've heard that data can actually end up anywhere on your disk, even if it's temporary. If data security is paramount, the whole disk should be crypted.

In my example, actually /bootdir should in the end probably not be mounted at all, or be mounted read-only and only be made writable/available if you're updating a kernel.

dbi · #5 March 21st, 2010, 16:40

OK, I agree that if data security is paramount, the speed doesn't matter so much, but in that case data integrity is of big importance too.

My understanding is that ZFS has the ability to provide applications with correct data even if the underlying device returns corrupted data (e.g. data was written in a sector that went "bad"). If the checksum doesn't match, ZFS gets the data from elsewhere e.g. from another disk in case of mirroring, from a different location (zfs set copies=N) or reconstructs it from other raid-z chunks. Along with passing the correct data ZFS corrects the on-device error. Hence the term "self-healing" [1].

So, the question is will ZFS loose its self-healing abilityif if it is NOT residing directly on the block device?

[1] http://2007.asiabsdcon.org/papers/P16-slides.pdf, pages 9-11

Please, note the "zfs slides presentation" is by Mr. Pawel Jakub Dawidek, not some random web source.

frijsdijk · #6 March 21st, 2010, 20:41

I can't really answer that. I would say that it doesn't providing we have a mirror or N > 1. Doesn't a .eli device act just like a block device?

I don't have a full understanding of the underlying layers and techniques to give a proper answer here..

dbi · #7 March 21st, 2010, 21:49

frijsdijk, please, don't feel like you have to give all the answers. It's just that your very nice how-to brought an interesting subject to my mind. So, I thought it would be nice to share it and hope someone who knows well the storage subsystem internals would make the things clear for me and everyone who might ask the same question in the future.

fgordon · #8 March 29th, 2010, 07:54

No nothing is lost of the ZFS capabilities as geli is transparent and running at a lower level than zfs - the only real effect is lower speed due to cryptography.

ilaiho · #9 June 1st, 2010, 10:23

What about encrypted swap and hibernate/resume?

dbi · #10 June 1st, 2010, 10:57

Encrypting swap is easy. See the examples in man geli.
Hibernate/resume highly depends on the hardware on which FreeBSD runs.
IMHO FreeBSD is vary bad in this area compared to other OSes.

ilaiho · #11 June 1st, 2010, 15:11

Quote:

Originally Posted by dbi

Encrypting swap is easy. See the examples in man geli.
Hibernate/resume highly depends on the hardware on which FreeBSD runs.
IMHO FreeBSD is vary bad in this area compared to other OSes.

I am just wondering, can FreeBSD resume from that like of encryted swap altogether. On Linux, swap encryption typically generates a random key on boot. Since the key always differs, you can't resume from that swap partition.

There are, however, ways to work around that, like storing both your data and swap partitions on an encrypted LVM.

zbroyar · #12 February 20th, 2012, 14:12

Probably, I'm doing something wrong: after completing all steps my laptop reboots after entering GELI password. And it's not panic, ~~b/c~~ because it happens so fast that I'm unable to catch the problem :-(

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
ZFS RAID: Disk fails while replacing another disk	hessi	General	1	March 3rd, 2010 17:19
Disk Encryption Software - cross platform	FBSD	General	1	November 18th, 2009 22:01
OpenVPN and Samba After Disk Full	abfan1127	Web & Network Services	2	April 7th, 2009 16:29
gmirror: lost disk, HOWTO proceed?	uisge	General	2	March 16th, 2009 21:26
g_vfs_done() Error GEOM Disk Encryption	ter2007	Installing & Upgrading	4	December 31st, 2008 17:26

The Following User Says Thank You to frijsdijk For This Useful Post:
dbi (March 21st, 2010)