Solved More advanced GELI setup

M

maximusmusterus1

I have finally decided to go with FreeBSD for my server, but I'm still about to clear some questions.

I want to use ZFS in combination with GELI, which is a pretty common thing I guess since there are plenty of resources. I managed to set up an encrypted pool that works fine. There are only 2 litte questions that remain.

I currently have 10 disks. Each disk has a GPT with one freebsd-zfs partition that is labeled with diskX. Then I added the GELI layer which results in diskX.eli devices that are subsequently combined to a zfs pool.
Additionally there is a root pool which contains the OS, that is encrypted as well (set up via automated installer). The root disk is protected by a passphrase (secret) and a keyfile (public on unencrypted /boot, just as a salt). The root disk contains key files for each pool disk. So the idea is that you only have to unlock root and the other disks get unlocked after that. This should be enough for reasonable security.

The root pool get decrypted on boot via loader, the other disks I currently decrypt by running a script after the system has booted. I will automate this with a rc.d script but there might be a better solution for this. This is the reason for asking here.

The other thing I thought of, is having a secondary passphrase. I might think of something as a failsafe, since the current passphrase is only in my memory and pretty complex. So I want to add something like a long sequence of words that are random but that can be reconstructed in some way. So question is, it is possible (and reasonable) to add a second passphrase to GELI?

Thanks for your time!
 
Hi maximusmusterus1 and welcome to the forums.

Firstly,
maximusmusterus1 said:
The root pool get decrypted on boot via loader, the other disks I currently decrypt by running a script after the system has booted. I will automate this with a rc.d script but there might be a better solution for this. This is the reason for asking here.
Click to expand...
The good news is that someone already wrote a script for this. From the "Encrypting Disk Partitions" section in the FreeBSD Handbook:
A rc.d script is provided to simplify the mounting of geli-encrypted devices at boot time. For this example, add these lines to /etc/rc.conf:
Code: 
geli_devices="da2"
geli_da2_flags="-k /root/da2.key"
This configures /dev/da2 as a geli provider with a master key of /root/da2.key. The system will automatically detach the provider from the kernel before the system shuts down. During the startup process, the script will prompt for the passphrase before attaching the provider.
Click to expand...

Secondly,
maximusmusterus1 said:
So question is, it is possible (and reasonable) to add a second passphrase to GELI?
Click to expand...
Yes (and yes). Have a look at the setkey option in the geli(8) man page and also the examples, at least one of which demonstrates more or less what you need:
Create an encrypted provider, but use two User Keys: one for your
employee and one for you as the company's security officer (so it is not
a tragedy if the employee "accidentally" forgets his passphrase):

# geli init /dev/da2
Enter new passphrase: (enter security officer's passphrase)
Reenter new passphrase:
# geli setkey -n 1 /dev/da2
Enter passphrase: (enter security officer's passphrase)
Enter new passphrase: (let your employee enter his passphrase ...)
Reenter new passphrase: (... twice)

You are the security officer in your company. Create an encrypted
provider for use by the user, but remember that users forget their
passphrases, so backup the Master Key with your own random key:

# dd if=/dev/random of=/mnt/pendrive/keys/`hostname` bs=64 count=1
# geli init -P -K /mnt/pendrive/keys/`hostname` /dev/ada0s1e
# geli backup /dev/ada0s1e /mnt/pendrive/backups/`hostname`
(use key number 0, so the encrypted Master Key will be re-encrypted by this)
# geli setkey -n 0 -k /mnt/pendrive/keys/`hostname` /dev/ada0s1e
(allow the user to enter his passphrase)
Enter new passphrase:
Reenter new passphrase:
Click to expand...
 
Thank you for your help, really appreciated! I've seen this example but somehow 'ignored' it because I thought it would not work with GPT labels - but after some searching I found out the syntax for this too. Thanks for putting me on the right path. (The system has multiple controllers and my last operating system initialized the controllers in different order at each boot so the device numbering was not always as expected. Don't know if this is a problem with FreeBSD too, but I better make sure it works by using labels.)

I will try the additional key, thanks for that too!
 
The great thing about the FreeBSD GEOM framework for storage is that you mostly don't need to care what is providing storage. Whether it is a whole disk, a BSD slice, a BSD partition, a GPT partition or an encrypted container it can be treated in the same way.

Using GPT labels is definitely sensible in general and certainly for GELI containers that are attached using configuration in /etc/rc.conf. Unless something has changed since I last looked at it though, GELI containers mounted before the root filesystem is mounted are best referred to using the /dev/da0p2 syntax in since all available devices are checked for GELI boot flags and the devices in /dev are checked before the devices in /dev/gpt. If the device numbering changes between boots that could cause you issues. You might want to do some testing if this is going to be a production server.

Talking of which, you mention you require a passphrase to decrypt the root partition, which means you will need someone to enter that passphrase at the console every time the server restarts. If this is a LAN fileserver sitting in your office it's probably not an issue but if it is sitting miles away in a server farm that someone needs to visit every reboot or in the event of a power cut you may want to rethink. What are you seeking to protect with full disk encryption? There is nothing confidential about the FreeBSD base system and utilities -- they are publicly available. Some configuration files may require protection and databases containing customer details certainly will but it may not be necessary to encrypt all your storage to achieve the security you need.
 
asteriskRoss said:
Using GPT labels is definitely sensible in general...
Click to expand...

Do it. I while back I set up a new server with a two-disk, 3Tb mirror. One of its uses is as a backup target for my laptop. There's a 2.5-inch drawer I can slide my laptop drive into when I need to restore a backup. I did this for the first time this morning and, having neglected to label the partitions on my 3Tb disks, I'm now looking at a 4-hour resilver after the drive order got bungled. :p It's only an inconvenience, but one that's easily avoided.
 
asteriskRoss, you mentioned some good points here, and I already put some thoughts into this myself. While the server is physically accessible (it's a local server) and attached to an UPS there might be situations where it has to be booted via remote access. As a possible solution I thought of something like setting up a small unencrypted machine running sshd and connecting it via serial TTY. If I'm not wrong, this should let me enter the passphrase remotely.

So why encrypting the root file system in the first place? I admit, that you're probably right, that this is only a small gain in terms of security, especially since I could use jails for most applications and let them reside on the encrypted disks. But I somehow like the idea that whoever gets access to the system has pretty much a unusable device with no clue what it was used for and what might be stored on it. (At least if he didn't know or crack the passphrase, is able to dump the memory while the system was running, etc.) With the root filesystem exposed, I could think of someone with enough time and knowledge might be get information out of temporary files, caches, log files, configuration, etc.

ANOKNUSA, I will certainly stick to labels. :p
 
maximusmusterus1 said:
With the root filesystem exposed, ...someone... might be get information out of temporary files, caches, log files, configuration, etc.
Click to expand...

You can partially mitigate this with cleaning /tmp, swap encrypting and putting your /usr/local/etc on the encrypted pool. Don't forget that with ZFS you can have totally diferent hierarchies for storage and filesystem, for example
Code: 
zpool system mountpoint=/
zpool data canmount=off
   data/etc mountpoint=/usr/local/etc
   data/var mountpoint=/var
 
Thank you for your ideas!
One more question… As mentioned, I set up the root on ZFS/GELI with bsdinstall. It uses da0p4 as device name, which causes boot problems in case a HDD is present in the hot-swappable bay I use for backups. Removing the disk prior to boot helps but this is not an ideal solution. I assume, as mentioned before, that the numbering of devices gets mixed up due to the multiple controllers. I inspected /boot/loader.conf and found the entries:
Code: 
geli_da0p4_keyfile0_load="YES"
geli_da0p4_keyfile0_type="da0p4:geli_keyfile0"
geli_da0p4_keyfile0_name="/boot/encryption.key"
etc.

Is it sufficient to change it to geli_da0p4 to geli_gpt_MYLABEL etc.? Or do I have to edit this anywhere else?
 
I don't believe you can use GPT labels to decrypt the partition during boot, but I could be wrong. You could give it a try; unless you've forgotten your passphrase there's no danger in it.

Ideally, though, you probably shouldn't have your backup drive in the hot swap bay all the time.
 
It is possible to attach the GELI container during boot based on GPT label... however it is annoying to do so as you have to tolerate failed attaches of the same device referred to in a different way. This is what I was referring to in my previous post:
asteriskRoss said:
Unless something has changed since I last looked at it though, GELI containers mounted before the root filesystem is mounted are best referred to using the /dev/da0p2 syntax in since all available devices are checked for GELI boot flags and the devices in /dev are checked before the devices in /dev/gpt. If the device numbering changes between boots that could cause you issues. You might want to do some testing if this is going to be a production server.
Click to expand...
The way GELI attaches containers at boot doesn't seem to be very smart. I believe (though let me say at this point that I have not examined the source code) it searches all the available devices for GELI containers that have their boot flags set and tries to attach any that aren't already. However, a GPT partition can be reached in several ways. What I would expect is that GELI will first try to attach the GELI container it finds in /dev/da0p4. If it doesn't successfully do so, it will try to attach the GELI container it finds in /dev/gptid/[identifier]. Only if it fails to attach will it attempt to attach the GELI container it finds in /dev/gpt/[label].

A reasonable workaround for you would be to configure your GELI key to be for the normal device number, the changed device number and the GPT label (in case your drive order is really messed up) in /boot/loader.conf. I'm pretty sure it is the boot flag and only the boot flag on the GELI container that counts, so unused or incorrect lines in /boot/loader.conf won't matter. So your contents might be something like:
Code: 
# Normal device number
geli_da0p4_keyfile0_load="YES"
geli_da0p4_keyfile0_type="da0p4:geli_keyfile0"
geli_da0p4_keyfile0_name="/boot/encryption.key"

# Reordered device number
geli_da1p4_keyfile0_load="YES"
geli_da1p4_keyfile0_type="da1p4:geli_keyfile0"
geli_da1p4_keyfile0_name="/boot/encryption.key"

# Ensure you can definitely mount with the GPT label
# and tolerate failed mounts
geli_gptlabel_keyfile0_load="YES"
geli_gptlabel_keyfile0_type="gpt/label:geli_keyfile0"
geli_gptlabel_keyfile0_name="/boot/encryption.key"
If I remember correctly, the names of the variables don't actually matter (you could use geli_foobar_keyfile0_{load,type,name}) as it is the contents of the the geli_X_keyfileX_type variable that associates the device with the key.
 
Thank you, for your replies. So in case I understand this correctly, it should not be a problem at all.

I thought the issue was something like that: At installation the root partition was da0p4. Lets now assume, that another controller/disk will be detected prior that, so the root partition is now da1p4 and some random device is at da0 (most likely to not even have da0p4). But GELI will prompt for da0p4 (which is non existent) and the progress will fail with "incorrect passphrase".

But first, da0p4 is the only GELI container with boot flag. So according to you, the boot process should ignore anything else anyway. And second, even if the device order is messed up and da0p4 does not exist, it will try to find the device via gptid?
 
maximusmusterus1 said:
Lets now assume, that another controller/disk will be detected prior that, so the root partition is now da1p4 and some random device is at da0 (most likely to not even have da0p4). But GELI will prompt for da0p4 (which is non existent) and the progress will fail with "incorrect passphrase".
Click to expand...

This is actually still possible, even with partition/filesystem labels. But being part of the GEOM disk framework, GELI can detect when a device has multiple nodes assigned to it. If it can't find the device letter and number you list in loader.conf, it at least tries to prompt you for the passphrase for the corresponding label and disk ID for that disk instead. I can't remember the exact warning, but the boot console output will show a message indicating that the disk previously had a different letter assignment---something like "Mounting root from /dev/gpt/<label> (was ada0p2)."

I've had problems with it before, but that was months ago, and it mysteriously resolved on its own. I know that might not be helpful, but it's something, I guess. :p
 
maximusmusterus1 said:
some random device is at da0 (most likely to not even have da0p4). But GELI will prompt for da0p4 (which is non existent) and the progress will fail with "incorrect passphrase".
Click to expand...
What I would expect is that you will have a prompt for any device hosting a GELI container with the boot flag set, irrespective of what keyfile configuration you add to /boot/loader.conf. If a device doesn't host a GELI container, you won't receive a prompt. The keyfile configuration is there so that GELI can combine the correct keyfile with the passphrase you enter.

So in your example, if your GELI container is normally hosted on /dev/da0p4 but when you add a drive it becomes /dev/da1p4, at boot you would receive a prompt only for da1p4. It's been a while since I played around with this and I certainly don't claim to be infallible so please do test to check. It's not me that is going to have to read the hate-mail from your users if your production server goes down :)
 
At the moment, it's just me trying to get things set up and playing around, so no worries about hate mail I guess. :D And this won't be a large online server with thousands of users – it's just for a small workgroup and in most cases I can easily access this thing. But you know, most likely it will fail when I am on vacation or something, so I just try to get a basic understanding of what can fail and how it can be solved or prevented. So I will certainly do some more tests, what happens when I put the drives in different slots etc.

Thank you guys a lot for your help and insights!
 
Short question:
For some reason GELI detached some disks leaving my pool in a degraded state. As I checked dmesg I found out they were detached on last close. Last thing I did before was starting zfs scrub – don't know if this might be related.
Anyway, I checked the documentation about this and think I can prevent this by adding geli_autodetach="YES" to rc.conf. But will this prevent the disks from being detached on shutdown too?
(Reattached the disks for now and ZFS did some resilvering with no permanent data errors)
 
Hi there....
How can I use zfs in freebsd without entering passphrase in every time want to boot system?
I use these commands but nothing happened:
geli configure -B prov
and -P
geom_eli_passphrase_prompt="NO" in /boot/loader.conf
Have any ideas?
 
if someone has physical access to appliance who can move the appliance 's hard drive to another system (computer with Windows os) to view and does everything.
I want to protect it in this case.
 
You must log in or register to reply here.
Back
Top