Solved [HELP] Unable to install FreeBSD with encrypted root (GELI/UFS)

Apologies if the title was vague, I'll try my best to explain. I'm new to the UNIX booting process and the components of such systems at this level.

Resources I'm using:
  1. https://forums.freebsd.org/threads/...stallation-how-to-with-video-if-needed.83934/
  2. https://www.c0ffee.net/blog/freebsd-full-disk-encryption-uefi
  3. https://forums.freebsd.org/threads/...oot-with-encrypted-zfs-root-using-geli.51393/
  4. View: https://www.youtube.com/watch?v=s1WvmIW_b8o
  5. https://docs.freebsd.org/en/books/handbook/bsdinstall/#bsdinstall-part-manual
  6. https://docs.freebsd.org/en/books/handbook/boot/#boot-introduction
So far, this is what I've done:

sh:
# Clean up if any partitions exist
gpart destroy -F ada0
gpart create -s GPT ada0

# Add EFI partition
gpart add -t efi -a 4K -s 100M -l EFI ada0
newfs_msdos -L EFI ada0p1

# ADD UFS boot partition
gpart add -t freebsd-ufs -a 4K -s 512M -l UFSBOOT ada0
newfs -L UFSBOOT ada0p2

# Create GELI-encrypted rootfs with just password
gpart add -t freebsd-ufs -a 4K -l ROOTFS ada0
geli init -b -e AES-XTS -l 256 /dev/ada0p3
geli attach /dev/ada0p3
newfs -L ROOTFS /dev/ada0p3.eli

# Mount partitions
mount -t ufs /dev/ada0p3.eli /mnt
mkdir -p /mnt/boot/
mount -t ufs /dev/ada0p2 /mnt/boot
mkdir -p /mnt/boot/efi
mount -t msdosfs /dev/ada0p1 /mnt/boot/efi
mkdir -p /mntboot/efi/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.EFI
cp -r /boot/* /mnt/boot/
cd /mnt
tar xvJf /usr/freebsd-dist/base.txz
tar xvJf /usr/freebsd-dist/kernel.txz
tar xvJf /usr/freebsd-dist/src.txz

# Create fstab entries to mount both rootfs and boot partition
rm /tmp/bsdinstall_etc/fstab
echo "/dev/ada0p3.eli    /        ufs    rw,ealgo=AES-XTS,len=256    0    0" >> /tmp/bsdinstall_etc/fstab
echo "/dev/ada0p2        /boot    ufs    rw                          1    1" >> /tmp/bsdinstall_etc/fstab

I get stuck after this every time because almost every guide now tells me to edit /etc/fstab and /boot/loader.conf, both of which are on the ISO (which is read-only, as the error that I get rightly says). I tried copying over loader.conf to /mnt/boot but that doesn't seem to do anything.

What do I do now? I feel like I'm missing some fundamental knowledge of the FreeBSD booting process which is the reason I'm stuck here. I would like to stick with the more traditional /boot and / and use UFS, but other than that I don't have many preferences.

Could someone also answer a question of mine while they're at it? From what I understand, the UEFI of the motherboard (or in this case, whatever my virtualization solution uses) looks for a FAT32 formatted GPT partition with a certain label ( /dev/ada0p1 in my case). Once it does find it, does it then start looking for a loader.conf? And how would it find this file if the partition isn't even mounted on to /boot yet? And that isn't possible unless the "root" partition is also mounted (unless I missed something). I am really confused now :)

Thanks!
 
There is no need for this step.
Code:
# ADD UFS boot partition
gpart add -t freebsd-ufs -a 4K -s 512M -l UFSBOOT ada0
newfs -L UFSBOOT ada0p2
loader.efi(8) is capable to load the kernel from a full encrypted partition, no need for a unencrypted boot partition.

Code:
# Create GELI-encrypted rootfs with just password
geli init -b -e AES-XTS -l 256 /dev/ada0p3
You need to initialize the provider with the "-g" option. geli(8)

Code:
                -g                Enable booting from this encrypted root
                                  filesystem.  The boot loader prompts for the
                                  passphrase and loads loader(8) from the
                                  encrypted partition.
For example
Code:
geli init -g -l 256 -s 4096 ada0p3

I get stuck after this every time because almost every guide now tells me to edit /etc/fstab and /boot/loader.conf, both of which are on the ISO (which is read-only, as the error that I get rightly says).
This is a misunderstanding on your side. Both files which needs editing are those on the newly created file system. In your case /mnt/etc/fstab and /mnt/boot/loader.conf, not the ones on the installation media.


For contemporary instruction follow link down below to install full root-on-UFS on geli(8) configured partition.

. https://forums.freebsd.org/threads/is-full-disk-encryption-with-ufs-possible.92399/post-643920

Skip the FreeBSD BIOS bootstrap code partition and bootcode code installation.

If you wish you can switch the freebsd-ufs and freebsd-swap partition creation. First swap, then UFS. Make sure to adapt /etc/fstab accordingly.
 
Honestly unless I'm missing something I don't understand why all the hassle. The 14.1 installer on the ISO is perfectly capable to install a fully GELI-encrypted system with root on ZFS without the need of doing anything and i can testify that is really foolproof. I installed my system this way and I had never touched FreeBSD before that. I suspect that this can be done with UFS too but I have no direct proof of that.
 
There is no need for this step.
Code:
# ADD UFS boot partition
gpart add -t freebsd-ufs -a 4K -s 512M -l UFSBOOT ada0
newfs -L UFSBOOT ada0p2
loader.efi(8) is capable to load the kernel from a full encrypted partition, no need for a unencrypted boot partition.

Code:
# Create GELI-encrypted rootfs with just password
geli init -b -e AES-XTS -l 256 /dev/ada0p3
You need to initialize the provider with the "-g" option. geli(8)

Code:
                -g                Enable booting from this encrypted root
                                  filesystem.  The boot loader prompts for the
                                  passphrase and loads loader(8) from the
                                  encrypted partition.
For example
Code:
geli init -g -l 256 -s 4096 ada0p3


This is a misunderstanding on your side. Both files which needs editing are those on the newly created file system. In your case /mnt/etc/fstab and /mnt/boot/loader.conf, not the ones on the installation media.


For contemporary instruction follow link down below to install full root-on-UFS on geli(8) configured partition.

. https://forums.freebsd.org/threads/is-full-disk-encryption-with-ufs-possible.92399/post-643920

Skip the FreeBSD BIOS bootstrap code partition and bootcode code installation.

If you wish you can switch the freebsd-ufs and freebsd-swap partition creation. First swap, then UFS. Make sure to adapt /etc/fstab accordingly.
My apologies for missing the -g flag from the manpage. It works now without a separate boot partition. Other than tradition, are there any specific advantages with using a separate /boot?

I'm facing a few other problems though:

1. Even though I set a root password during setting all of this up in the LiveCD, I never get asked for it after the prompt to unlock the GELI provider. Why is that?
2. poweroff makes the system hang after detaching the GELI provider. What am I missing?

Thank you for your answer and for the link, unfortunate that I didn't find it before.
 
Honestly unless I'm missing something I don't understand why all the hassle. The 14.1 installer on the ISO is perfectly capable to install a fully GELI-encrypted system with root on ZFS without the need of doing anything and i can testify that is really foolproof. I installed my system this way and I had never touched FreeBSD before that. I suspect that this can be done with UFS too but I have no direct proof of that.
I don't want to use ZFS and I do not see an option for an encrypted root with UFS in the installer (maybe I didn't spot it).

Also, I'd like to learn more about the BSDs and UNIX-like OSes in general. I work in IT but I never deal with operating system internals in my line of work and I'd like to learn for my own sake. It gives me a better sense of control over my installation which is a feeling I've never had before regarding my operating system.

Also, would you happen to know about the last question I asked in my OP? How does the UEFI know where to find the boot files?
 
Also, would you happen to know about the last question I asked in my OP? How does the UEFI know where to find the boot files?
You or the installation script can set EFI vars with efibootmgr(8). There, you can specify what disk, what partition and what loader file including its path.

Otherwise, you have some default files, depending the arch, that are searched in each EFI partition. For amd64, it's /efi/boot/bootx64.efi.

Is this answers to your question?
 
1. Even though I set a root password during setting all of this up in the LiveCD, I never get asked for it after the prompt to unlock the GELI provider. Why is that?
I'm not sure I understand. Do you mean at the login prompt for the root user account after the boot process has finished?

2. poweroff makes the system hang after detaching the GELI provider. What am I missing?
Which FreeBSD version have you installed?

Check sysctl hw.efi.poweroff. This should be "1"

Also, would you happen to know about the last question I asked in my OP? How does the UEFI know where to find the boot files?
Could someone also answer a question of mine while they're at it? From what I understand, the UEFI of the motherboard (or in this case, whatever my virtualization solution uses) looks for a FAT32 formatted GPT partition with a certain label ( /dev/ada0p1 in my case). Once it does find it, does it then start looking for a loader.conf? And how would it find this file if the partition isn't even mounted on to /boot yet? And that isn't possible unless the "root" partition is also mounted (unless I missed something). I am really confused now
The bootstraping process is as follows: The UEFI firmware is looking for a OS loader in the EFI System Partition (ESP). For FreeBSD this is loader.efi(8) (renamed /efi/boot/bootx64.efi or /efi/freebsd/loader.efi). When found by the UEFI firmware and loaded, loader.efi(8) begins with the FreeBSD system bootstrap procedure.

For detailed information see uefi(8) and loader.efi(8).

In case of geli(8) encrypted root partition (provider), after the UEFI firmware has loaded loader.efi(8), loader.efi(8) prompts for the geli(8) passphrase to unlock the encrypted root partition (provider). Now that the provider is unlocked and the file system is readable the boot process continues, pausing at the boot menu for possible user intervention.

If you escape to loader prompt at the boot menu, ls will show the entire file system.

Enter at the loader prompt "?" to show other loader built-in commands. See loader_simp(8) for details.
 
The 14.1 installer on the ISO is perfectly capable to install a fully GELI-encrypted system with root on ZFS without the need of doing anything ... I suspect that this can be done with UFS too
The FreeBSD installer doesn't provide a guided geli(8) encrypted Root-on-UFS installation script as it does with Root-on-ZFS, it must be configured manually.
 
You or the installation script can set EFI vars with efibootmgr(8). There, you can specify what disk, what partition and what loader file including its path.

Otherwise, you have some default files, depending the arch, that are searched in each EFI partition. For amd64, it's /efi/boot/bootx64.efi.

Is this answers to your question?
Thanks, but I'm a bit confused. Does the UEFI just look for the string "efi/boot/bootx64.efi" and if it finds a file with this as a part of its path, it just uses it?
 
I'm not sure I understand. Do you mean at the login prompt for the root user account after the boot process has finished?
Yes.

Which FreeBSD version have you installed?
14-1 RELEASE. sysctl hw.efi.poweroff is indeed 1.

I did as you said and didn't make a separate partition for boot. I'm able to reboot the VM and log in as root into the installed system (which is great since I've been at this for a bit now). With that said, why does my install fail when I have a separate /boot partition? I have never managed to get that running till now. I understand that perhaps there is no reason to have a separate partition any more but I'm asking this out of academic interest.

Thanks for your time. A picture of the moment when I hit poweroff is attached.
 

Attachments

  • poweroff.png
    poweroff.png
    123.2 KB · Views: 46
Thanks, but I'm a bit confused. Does the UEFI just look for the string "efi/boot/bootx64.efi" and if it finds a file with this as a part of its path, it just uses it?
Yes, it's an PE32+ executable. It loads and runs it. It's probably a few complex than that (for example, I think it must check if this file is really an executable before to run it), but the details depend on the implementation of UEFI in each machine.
 
Yes, it's an PE32+ executable. It loads and runs it. It's probably a few complex than that (for example, I think it must check if this file is really an executable before to run it), but the details depend on the implementation of UEFI in each machine.
Thanks
 
I'm not sure I understand. Do you mean at the login prompt for the root user account after the boot process has finished?
I can't tell, works for me.

2. poweroff makes the system hang after detaching the GELI provider. What am I missing?
I'm able to reboot the VM
Are you using VirtualBox as virtualizer? Because the problem looks exactly like what occurs with FreeBSD guests running on vbox.

To correct the issue, go to the VM's "Settings -> System -> Motherboard: Chipset", change from "PIIX3" to "ICH9".

Apropos, instead of poweroff(8), ACPI shutdown the VM with "Host + H" keys.

With that said, why does my install fail when I have a separate /boot partition?
Difficult to say without further configuration details, error messages. A seperate unencrypted boot partition and a encrypted root partition is perfectly doable.
 
Thank you everyone. These are the commands I used to get this running with an encrypted root (no separate boot partition) with GELI (commands for future reference):

Code:
# Clean up if any partitions exist
gpart destroy -F ada0
gpart create -s GPT ada0

# Add EFI partition
gpart add -t efi -a 4K -s 100M -l EFI ada0
newfs_msdos -L EFI ada0p1

# Create GELI-encrypted rootfs with just password
gpart add -t freebsd-ufs -a 4K -l ROOTFS ada0
geli init -b -e AES-XTS -l 256 /dev/ada0p2
geli attach /dev/ada0p2
newfs -L ROOTFS /dev/ada0p2.eli

# Mount partitions
mount -t ufs /dev/ada0p2.eli /mnt
mkdir -p /mnt/boot/efi
mount -t msdosfs /dev/ada0p1 /mnt/boot/efi
mkdir -p /mnt/boot/efi/EFI/BOOT
cp /boot/loader.efi /mnt/boot/efi/EFI/BOOT/BOOTX64.EFI

cd /mnt
tar xvJf /usr/freebsd-dist/base.txz
tar xvJf /usr/freebsd-dist/kernel.txz
tar xvJf /usr/freebsd-dist/src.txz

# Create fstab entries to mount both rootfs and boot partition
chroot /mnt
echo "/dev/ada0p2.eli    /    ufs    rw    0    0" >> /etc/fstab

echo "geom_eli_load=\"YES\"" >> /boot/loader.conf
echo "vfs.root.mountfrom=\"ufs:ada0p2.eli\"" >> /boot/loader.conf

Reboot after this, it should work (please confirm if escaping the characters works properly if you're seeing this in the future, if not comment under this with your commands)
 
I can't tell, works for me.



Are you using VirtualBox as virtualizer? Because the problem looks exactly like what occurs with FreeBSD guests running on vbox.

To correct the issue, go to the VM's "Settings -> System -> Motherboard: Chipset", change from "PIIX3" to "ICH9".

Apropos, instead of poweroff(8), ACPI shutdown the VM with "Host + H" keys.


Difficult to say without further configuration details, error messages. A seperate unencrypted boot partition and a encrypted root partition is perfectly doable.
Yes, I'm using VirtualBox. Thank you
 
On topic of full disk encryption, what cipher/cipher len does the default installer use when selecting full disk encryption via iso boot ?
 
On topic of full disk encryption, what cipher/cipher len does the default installer use when selecting full disk encryption via iso boot ?
Encryption algorithm: AES-XTS (default when not set explicitly)
Data Key length to use with the given cryptographic algorithm: 256 (default 128)
Decrypted provider's sector size: 4096

/usr/libexec/bsdinstall/zfsboot
Code:
201 GELI_PASSWORD_GELIBOOT_INIT='geli init -bg -e %s -J - -l 256 -s 4096 "%s"'
For AES-XTS search in file.
 
Encryption algorithm: AES-XTS (default when not set explicitly)
Data Key length to use with the given cryptographic algorithm: 256 (default 128)
Decrypted provider's sector size: 4096

/usr/libexec/bsdinstall/zfsboot
Code:
201 GELI_PASSWORD_GELIBOOT_INIT='geli init -bg -e %s -J - -l 256 -s 4096 "%s"'
For AES-XTS search in file.
Thanks, is there a way to modify those parameters during the install ? Or is the option to install manually ?
 
... is there a way to modify those parameters during the install ?
It's not possible during install (as I know off). Certainly the zfsboot script doesn't support dropping at the "Partitioning" menu to "Shell" and configure a geli(8) provider for Root-on-ZFS.

Or is the option to install manually ?
That, or modify zfsboot to your liking.

Very easy with .img installation images: dd(1) to USB, mount file system read/write, edit file.

To customize zfsboot on .iso installation images needs a little more work, but not much. Extract ISO, edit file, build new ISO. A tutorial can be found in section "BUILDING AUTOMATIC INSTALL MEDIA" at end of the bsdinstall(8) manual. This requires FreeBSD source installed for making iso images and other dependency scripts.
 
Back
Top