UFS UEFI booting with separate /boot partition

abefar · Mar 14, 2016

I am in the process of migrating to UEFI on my laptop. I would like /boot to be on a separate partition in order to have / on GELI, but I cannot get the UEFI bootloader to play along.
It finds both my mirrored /boot UFS partitions just fine, but refuses to continue because /boot/loader.efi does not exist.

I tried renaming the existing /boot/boot file (a piece of MBR bootcode, I presume) and creating a boot folder instead, containing a hard link to loader.efi. After rebooting I was dropped into a bootloader prompt, and using boot /kernel/kernel it actually booted the kernel.
However, the hardlinking trick did not work for loader.conf, so none of the required geom_ modules got loaded.

Is there a way to instruct the bootloader to look in / instead of /boot/, without modifying the source code?
I can't seem to find information about this matter in uefi(8) or any other part of the EFI-related documentation.

asteriskRoss · Mar 15, 2016

Have a look at Thread 51393. It is written for ZFS on GELI but can be adapted for using UFS on GELI.

abefar said:
Is there a way to instruct the bootloader to look in / instead of /boot/, without modifying the source code?

Not that I am aware of; what you want was not configurable for FreeBSD 10.1 and I don't see any indication of a change in the release notes for FreeBSD 10.2. However, given your question I can have a guess at where your issue is. I suspect that you have put the boot files at the root of an unencrypted UFS partition that you have configured to mount at /boot in /etc/fstab. Instead you need to create a directory called boot on that partition to contain your boot files. The way to think about it is that from the perspective of the boot loader, your unencrypted UFS partition is a minimal root partition and therefore needs to be structured in the same way as a full root partition on a normal system.

Unfortunately, this also means you need to do some fudging to make /boot appear at the correct place from the perspective of your running system for kernel updates and other system administration. The way to do it is to configure /etc/fstab to mount the unencrypted UFS partition containing your boot files at something like /ufsboot and then create /boot as a symbolic link to /ufsboot/boot (so that cd /boot actually puts you in /ufsboot/boot).

If this doesn't solve your issue or it isn't clear what I'm suggesting you do, please post the output of gpart show and the contents of /etc/fstab and /boot/loader.conf and I'll try to offer more specific help.

abefar · Mar 15, 2016

asteriskRoss said:
Have a look at Thread 51393. It is written for ZFS on GELI but can be adapted for using UFS on GELI.

Not that I am aware of; what you want was not configurable for FreeBSD 10.1 and I don't see any indication of a change in the release notes for FreeBSD 10.2. However, given your question I can have a guess at where your issue is. I suspect that you have put the boot files at the root of an unencrypted UFS partition that you have configured to mount at /boot in /etc/fstab. Instead you need to create a directory called boot on that partition to contain your boot files. The way to think about it is that from the perspective of the boot loader, your unencrypted UFS partition is a minimal root partition and therefore needs to be structured in the same way as a full root partition on a normal system.

Unfortunately, this also means you need to do some fudging to make /boot appear at the correct place from the perspective of your running system for kernel updates and other system administration. The way to do it is to configure /etc/fstab to mount the unencrypted UFS partition containing your boot files at something like /ufsboot and then create /boot as a symbolic link to /ufsboot/boot (so that cd /boot actually puts you in /ufsboot/boot).

If this doesn't solve your issue or it isn't clear what I'm suggesting you do, please post the output of gpart show and the contents of /etc/fstab and /boot/loader.conf and I'll try to offer more specific help.

Thanks a lot! I did not consider this solution at all.

kpa · Mar 15, 2016

There is boot.config(5) that can be used at least with BIOS booting to modify the paths for loader(8) and other things. No idea how it works with UEFI.

asteriskRoss · Apr 5, 2016

Regarding kpa's suggestion, I see from the release announcement for FreeBSD 10.3:

The UEFI boot loader received several improvements: It now follows /boot/config and /boot.config files [...]

efbsd · Mar 29, 2025

I'm having issues with `boot` in a separate partition, when on the `loader.efi phase`, I get `ERROR: cannot open /boot/lua/loader.lua: no such file or directory`, I understand this issue is because the default directory for the loader is `/boot`, but I want to place all `boot` files in its partition `/` (So I can just mount the partition on root filesystem and not doing symlink/nullfs), is there a way I can change that `/boot` directory to just `/`? I've tried placing a `/boot.config` with `0:ada(0p2)/loader` to no avail

Partition scheme:

Code:

ada0     GPT

1        efi              /boot/efi

2        freebsd-ufs      /boot

3        freebsd-ufs      /

Inspecting the source code (https://github.com/freebsd/freebsd-src/tree/main/stand) makes me think this is hardcoded, is there something I'm missing?

T-Aoki · Mar 29, 2025

IIRC (it already was a long ago when I did kicking ZFS on Root from UFS boot partition to test early UEFI boot code when the boot1.efi on test somehow failed), you can force mounting root from where you really want by specifying vfs.root.mountfrom= line correctly in your /boot/loader.conf.

But unfortunately, IIRC, I couldn't mount the /boot into actual root partition (at the moment, /boot in actual root partition cannot be blank) at least at the moment. If making /boot in actual root partition blank and attempt to mount /boot partition on it with specifying /etc/fstab, boot failed, maybe because of the order that loader processes /boot/loader.conf.

efbsd · Mar 29, 2025

When I'm at the OK shell, I can boot fine with boot /kernel/kernel, vsf.root.mountfrom variable is also picked from loader.env so it continues to boot without issues. Think maybe the issues is between loader and loader.lua where it's using hardcoded paths.

T-Aoki · Mar 29, 2025

efbsd said:
When I'm at the OK shell, I can boot fine with boot /kernel/kernel, vsf.root.mountfrom variable is also picked from loader.env so it continues to boot without issues. Think maybe the issues is between loader and loader.lua where it's using hardcoded paths.

Is your loader.env exists in ESP, not in /boot partition?
If so, it could be too early to set vfs.root.mountfrom variable.
Possibly it makes first stage loader.efi (or boot1.efi) to look for loader.lua in /boot/ of the actual root partition, not in /boot partition.

efbsd · Mar 30, 2025

Yes, loader.env is in ESP, however before I added it it just made kernel prompt for root partition to boot on, no changes to what boot1.efiis looking.

I see some hardcoded paths on https://github.com/freebsd/freebsd-...fc765b28ee9c8bf41e8fc1b0261/stand/defs.mk#L72 & https://github.com/freebsd/freebsd-...b28ee9c8bf41e8fc1b0261/stand/lua/core.lua#L72, the later hints that it could be changed but doesn't specify where (or maybe it's just a build time config)

T-Aoki · Mar 30, 2025

Do you have everything that should be in /boot (on straightforward installation) in your /boot partition?

Looking into the code mentioned below,

efbsd said:
I see some hardcoded paths on https://github.com/freebsd/freebsd-...fc765b28ee9c8bf41e8fc1b0261/stand/defs.mk#L72 & https://github.com/freebsd/freebsd-...b28ee9c8bf41e8fc1b0261/stand/lua/core.lua#L72, the later hints that it could be changed but doesn't specify where (or maybe it's just a build time config)

if you really want to change this, defining LUAPATH in your /etc/src.conf (it can be overridden, as defined with ?=) when you build world (or at least in /usr/src/stand) locally.

And maybe setting lua_path in your /boot/loader.conf (which loader.efi actually looks into) would be needed. (As this hardcoded /boot/lua is in a conditional for failover when lua_path is not defined.)

T-Daemon · Mar 30, 2025

efbsd said:
I'm having issues with `boot` in a separate partition,

What exactly is your use case for a separate "/boot" partition?

Given the subject of the OP's opening posting, booting a geli(8) encrypted root file system, as far as I can tell, this seems to be the reason why you want a separate "/boot" partition.

Note that the FreeBSD boot loader is perfectly capable of booting from a geli(8) encrypted Root-on-UFS, including the kernel and the whole "/boot" directory.

See geli(8) manual for the "init and configure -g" option.

efbsd · Mar 30, 2025

Yes, the boot partition has everything that a normal boot folder would have.

I would prefer to not build a custom bootloader since it would add more complexity to the maintenance of the system.

I've set lua_path on both /loader.conf and /boot/loader.conf (even when I don't want to create a "fake" boot folder under this partition) but it's simply not picked up, the error message stills points to /boot/lua/loader.lua hinting that either loader.efi is not picking this variable or maybe it's just a build time config and cannot be changed run time.

efbsd · Mar 30, 2025

Yes, we are already using encrypted geli, however we want to remotely unlock it (via ssh), and since it's not natively supported we're working with this setup (https://github.com/emtiu/freebsd-outerbase).

Currently, we have boot on the outer base, but wanted to have it on a separate partition to make the setup more clear (esp, boot, outer-base, encrypted-inner-base) and also the mounts would be identical on both bases, which removes a bit of ambiguity.

Right now, we either need to symlink /outer/boot to /boot on the inner base (which give issues with some installers like OPNsense bootstrap script where the tar unpack overwrites the link) or mount as nullfs which then requires nullfs_load (which isn't much of an issue, but we still need to go through outer to access boot)

T-Daemon · Mar 31, 2025

efbsd said:
I would prefer to not build a custom bootloader

I believe there is no other way, the paths look hard coded:

/usr/src/stand/common/paths.h

Code:

#ifndef _PATHS_H_
#define _PATHS_H_

#include <sys/boot.h>   /* To get kernel path */

#define PATH_DOTCONFIG  "/boot.config"
#define PATH_CONFIG     "/boot/config"
#define PATH_LOADER     "/boot/loader"
#define PATH_LOADER_EFI "/boot/loader.efi"
#define PATH_LOADER_ZFS "/boot/zfsloader"
#define PATH_LOADER_CONF "/boot/loader.conf"
#define PATH_DEFAULTS_LOADER_CONF "/boot/defaults/loader.conf"

#endif /* _PATHS_H_ */

Rich (BB code):

% grep -r paths.h /usr/src/stand
/usr/src/stand/i386/isoboot/isoboot.c:#include "paths.h"
/usr/src/stand/i386/boot2/boot2.c:#include "paths.h"
/usr/src/stand/i386/gptboot/gptboot.c:#include "paths.h"
/usr/src/stand/i386/zfsboot/zfsboot.c:#include "paths.h"
/usr/src/stand/powerpc/boot1.chrp/boot1.c:#include "paths.h"
/usr/src/stand/efi/gptboot/proto.c:#include "paths.h"
/usr/src/stand/efi/loader/main.c:#include <paths.h>
/usr/src/stand/efi/boot1/boot1.c:#include "paths.h"
/usr/src/stand/efi/boot1/proto.c:#include "paths.h"
/usr/src/stand/kboot/kboot/hostdisk.c:#include <paths.h>

Code:

% strings /boot/loader.efi | grep '/boot'
/boot/lua/loader.lua
/boot/fonts/INDEX.fonts
/boot/lua/?.lua
/boot/fonts/8x16.fnt
/boot/lua
/boot/kernel;/boot/modules
/boot/defaults/loader.conf
/boot/lua/5.4/lib/?.so;/boot/lua/5.4/lib/loadall.so;./?.so
%s/boot/%s
/boot/fonts/8x16b.fnt
/boot/kernel/kernel

T-Daemon · Mar 31, 2025

efbsd said:
it would add more complexity to the maintenance of the system.

efbsd said:
Currently, we have boot on the outer base, but wanted to have it on a separate partition to make the setup more clear (esp, boot, outer-base, encrypted-inner-base)

As I see it, you are adding another layer of complexity to the setup.

What possible advantage could it bring putting outer-boot and outer-base on two partitions instead of one?

efbsd said:
Yes, we are already using encrypted geli, however we want to remotely unlock it (via ssh),

I had this setup tested before. IMO, the simplest setup to maintain is to create one complete outer-base/boot and a encrypted complete inner-base/boot, both the same version, of course.

T

Post in thread 'Unlock GELI partitions at boot through SSH'

Jan 31, 2025

freezr said:
The computer should arrive tomorrow... I really hope (better dreaming) the scripts work out-of-the-box...
It would be a shame fall-back on Linux just because it is easier...

Actually those scripts are not absolutely necessary.

Basically they install two systems side by side on one or multiple disk, one unencrypted, the other encrypted. This can also be done with the FreeBSD installer by running it twice on the booted installer system,

First install encrypted "Auto (ZFS) Guided Root-on-ZFS", create a large swap in size of "swap + unencrypted system", let's say 8G swap, 2G for...

efbsd · Mar 31, 2025

T-Daemon said:
As I see it, you are adding another layer of complexity to the setup.

What possible advantage could it bring putting outer-boot and outer-base on two partitions instead of one?

Since we're sharing boot between both bases it would just make things clear, also we would not need to mount outer first in order to access boot from inner. However as things are hardcoded this is not worth it until these paths are configurable run time (maybe in /boot.config or loader.env)

T-Daemon said:
I had this setup tested before. IMO, the simplest setup to maintain is to create one complete outer-base/boot and a encrypted complete inner-base/boot, both the same version, of course.

T

Post in thread 'Unlock GELI partitions at boot through SSH'

Jan 31, 2025

freezr said:

The computer should arrive tomorrow... I really hope (better dreaming) the scripts work out-of-the-box...
It would be a shame fall-back on Linux just because it is easier...

Click to expand...

Actually those scripts are not absolutely necessary.

Basically they install two systems side by side on one or multiple disk, one unencrypted, the other encrypted. This can also be done with the FreeBSD installer by running it twice on the booted installer system,

First install encrypted "Auto (ZFS) Guided Root-on-ZFS", create a large swap in size of "swap + unencrypted system", let's say 8G swap, 2G for...

T-Daemon

That would require 2 boot, which may be fine if using stock or having a single kernel, but for other setups it would eat up more space than it's actually needed (which could be limited in case of systems running with low storage).

The current shared boot works fine though, we'll have to stick with symlink/nullfs in the meanwhile.