Order / naming of devices being recognized + GELI

[sukosevato]

Hey,

I followed this guide, and I currently have the following:

2x 1 TB encrypted with GELI with a ZFS mirror file system on top of it.

Those two HDD's are detected as ada0 and ada1. And the OS boots from ada0p3 and ada1p3. All works fine.

However, when I wanted to add another eight 3 TB HDD's to my server to create a RAID-Z2 with, it refused to boot. The issue being that other HDD's got detected earlier and claiming ada0/ada1. GELI asked me for the pass phrase for ada2p3 and ada3p3, (which were the 1 TB disks). But it refused to accept it / the system did nothing.

After rearranging the SATA cables I managed to get the system to detect the two 1 TB's as ada0/ada1 again, because one of the SATA controllers was getting detected first. And GELI accepted my pass phrase again and the system booted fine.

But this isn't really a very convenient thing to do, having to rearrange the position of HDD's when you add other HDD's/controllers to ensure your GELI devices are still detected in the order you first installed them in.

In the /boot/loader.conf you have to specify which key file GELI should use for which device name and since the device name is different, GELI won't use the right key file and therefore fails to decrypt the device. Would it be possible to specify which device GELI should decrypt by using GEOM labels so the order in which the devices are detected is irrelevant because the GEOM labels are bound do the disk and not the order in which it is detected?

I spoke to the author of the tutorial I followed and he said the following:

"You can label up disks using geom_label - but this isn't present during
the boot loader/kernel loading process... so it's no use for a boot
disk. It's just something we have to work around to boot from encrypted
drives :-("

Would it be possible to get the GEOM label modules to load before the GELI modules are loaded so that this would be possible? Or is there another way to do this that is easier?

Thank you for your time.

Regards,

Sukosevato

 

[jem]

Perhaps GPT partition labels might be available to use earlier in the boot process. These are different from glabel labels.

 

[wblock@]

GPT labels are different, but it's the same kernel module that provides them as device labels. A workaround is to have an unencrypted boot partition, as shown in Thread 29652.

 

[sukosevato]

Thanks for pointing me to those GPT labels. Didn't know those existed as well. I've now created my RAIDZ2 pool over 8 x 3 TB with GPT labels. Works great. When I boot into a live CD, even after I connected every SATA cable to a different port it still worked fine. About the original issue, I'm going to reinstall in a month or 1-2. First going to learn some more on this test run. Will definitely test if it works for boot devices as well when doing the reinstall.

 

[Bucky]

Solution?

I think I have a solution to this issue. Background to see if my situation is on-point.

Building server box with FreeBSD 10.1x64. Install two 20GiB PATA drives to be mirrored as ZFS boot devices with geli encryption, using the regular install routine in 10.1 (this automatically makes 4 partitions of each drive, ada0p1-4 (and ada1...), with p4 being the encrypted part.) For this I use a passphrase to boot.

Note carefully, I use PATA drives as my boot drives (had them lying around, unused). The OS sees them as ada0 and ada1.

FreeBSD 10.1 installed. System powered down.

Now added a Promise TX4-300 controller card and connected 4 4TiB drives plus connected 3 more 4TiB drives to existing motherboard SATA ports. Total = 7 4TiB drives for NAS in raidz2.

Powered on.

Boot sequence now asks me to enter passphrase for ada4 and ada5, the OS having re-ordered the drive sequencing to recognize the TX4-300 controller SATA drives *ahead* of the PATA drives. But the boot drives are still recognized ahead of the other *motherboard* ported SATA drives.

The boot goes nowhere since /boot/loader.conf references ada0 and ada1 as the boot drives and hangs asking for the passphrase to ada4p4. Make a note of the new adaX numbers for the boot drives. (May have to hit the scroll-lock key so you can page backwards through the boot sequence to find the boot drive(s) /dev/ names.)

Power down. Disconnect the drives connected to the TX4-300 controller temporarily, then power up and reboot. The boot drives are back to ada0 and ada1.

Log in and edit /boot/loader.conf to change the adaX values referenced for the boot drive(s) *when* the TX4-300 controller has drives attached. My change looked like this:

geli_ada0p4_keyfile0_load="YES"   -> geli_ada4p4_keyfile0_load="YES"
geli_ada0p4_keyfile0_type="ada0p4:geli_keyfile0"   -> etc...
geli_ada0p4_keyfile0_name="/boot/encryption.key"   -> etc...

if you have mirrored boot drives, obviously change the adaX numbers for each drive.

shutdown -p now

Reconnect the TX4-300 drives and restart.

The passphrase request should now point to the correct, renamed boot drives.

Hope this helps someone else.