Different approaches to Boot Environments — which do you use?

J

jwdevel

I'm just getting started with BEs and ZFS. As I build my understanding, reading other people's tutorials, etc, I feel like there are a few distinct ways that BEs can be used to (eg) upgrade a system. Below is my current view — does anyone do anything different? I'm just trying to see what different options are out there, with different pros and cons.

(Aside: some threads on a similar topic: https://forums.freebsd.org/threads/boot-environments-beadm.87789/, https://forums.freebsd.org/threads/...ot-environments-with-make-installworld.84677/)

ZFS datasets
First, there is the question of how to arrange the ZFS datasets — the main ones being "shallow" or "deep", as described in the bectl(8) manpage. The older beadm tool seems to assume "shallow," and that's also what the installer uses when setting up boot-from-zfs. So, that's what I have.

▶ I wonder: Do many of you use the "deep" layout (or something else)? What advantages are there, other than being a more readable/obvious?

Upgrade Process

Second, there is the process used to actually perform an upgrade and roll back if needed.
It seems like there are roughly 4 approaches, but I wonder if any of you do something different?

Also, for all of these, I think there is still a separate concern of upgrading the BIOS/EFI boot loader, between releases? That stuff is outside of ZFS, so not covered by BEs, as I understand it.

Let's say we're currently in a "v99" BE and are upgrading to v100; here are the approaches as I see them:

A:
  1. create new BE called v100
  2. mount it somewhere (/tmp/v100)
  3. upgrade inside that mount, using appropriate flags (such as freebsd-update -b /tmp/v100 or make DESTDIR=/tmp/v100 ..., etc)
  4. unmount
  5. reboot into it, confirm all is well
  6. If something goes wrong, activate the old v99 BE.
Downsides: You must remember to use the special flags to target the dest dir. I would worry in the back of my mind that some tool assumes the wrong thing and touches the running system (rather than the /tmp/v100 dir) accidentally.

B:
This is essentially the same as (A), except rather than using special flags to do an upgrade in a dest directory, we would use either chroot or jail, and then use the "normal" upgrade commands inside there.

Example w/chroot: https://vermaden.wordpress.com/2021/02/23/upgrade-freebsd-with-zfs-boot-environments/

Downsides: You need some familiarity with chroot/jail. The bectl tool has built-in support for jail/unjail, though.

C:
  1. create new BE called v100
  2. reboot into it immediately
  3. upgrade the running system
  4. reboot again (to complete the upgrade), confirm all is well
  5. If something goes wrong, activate the old v99 BE.
Downsides: Two reboots

D:
  1. create new BE called v99-backup
  2. rename the current BE v99 to v100
  3. upgrade the running system
  4. reboot into it, confirm all is well
  5. If something goes wrong, activate the v99-backup BE.
  6. optionally, rename v99-backup to plain v99.
Downsides: You never actually booted v99-backup before an emergency situation; makes me slightly uneasy.

----

Most guides I've seen use approach (A) or (B). Personally, (C) appeals to me since it seems the most foolproof.

▶ Do you use one of those, or something different?
 
Well, it's simpler for me...

- Create a new BE ("backup BE").
- Upgrade on the default one.
- If something went wrong, boot on the "backup BE" (via the beasty menu).
- You can always rename the BEs, whatever BE you keep.

Personally, I have never understood why people use such complicated procedures as those you describe. Most of the time, everything goes well. In this last case, this method is the one that requires the least effort.
 
Emrion said:
Well, it's simpler for me...

- Create a new BE ("backup BE").
- Upgrade on the default one.
- If something went wrong, boot on the "backup BE" (via the beasty menu).
- You can always rename the BEs, whatever BE you keep.

Personally, I have never understood why people use such complicated procedures as those you describe. Most of the time, everything goes well. In this last case, this method is the one that requires the least effort.
Click to expand...

That's essentially (D) in my list, right? Unless I misunderstand you...
 
Deep, shallow, I take the defaults from install. Why? Usually simplier and easier in the long run.

I use pkgs and freebsd-update, so no compiling from source.
If I am staying within a version like 14-release-p[0-n] I use freebsd-update but I understand that the tool will create a new BE that represents the system BEFORE an upgrade. The upgrade happens into my CURRENT BE, when it's done (don't forget to update packages too) I reboot. I also use bectl rename to make sure the BE name reflects what it is. "default" is that 13-release, 14-release, 15-current?.

Now if I am going across versions, say 14.1-RELEASE to 14.2-RELEASE or 13.2-RELEASE to 14.2-RELEASE, I do the steps of manually creating a new BE, the doing all the chroot upgrades into that. freebsd-update and pkg commands have options to specify "I'm doing this into chroot".
Why? Because at the end of the process, I am still running in a coherent un-upgraded BE and I have a new BE that is the upgrade and has everything complete: base system and pkgs/ports. A quick bectl command to activate the new BE and a single reboot and you are up running in the new version.

I guess this is your "A" but it's only in specific instances.
I also vigorously prune my BEs, typically only have one that I am running, maybe the previous one while I verify the new one.

NOTE:
Going across versions be very very careful around "zpool upgrade" commands. Don't do zpool upgrade when you are on new version before you update bootcode. Why? zpool upgrade without updating bootcode may wind up in an unbootable system. New bootcode understands older zpool so can boot, but older bootcode may not understand newer zpool and can't boot.
 
mer said:
...

Going across versions be very very careful around "zpool upgrade" commands. Don't do zpool upgrade when you are on new version before you update bootcode.
Click to expand...

Thanks for mentioning this. I'm just learning about the (not great) current state of affairs here.
For anyone else's reference, there's a bug about the poor state of documentation for updating bootcode (#255318, open since 2021), and bugs around EFI+ZFS only installing the boot code onto one of the disks, so you can't boot if it fails (#262770 open since 2022, #229191 since 2018).

In my case, I have a 3 disk raid for zfsroot, and it is non-UEFI (BIOS, but still GPT). Does anyone know the proper way to update it, after an upgrade? I guess this wiki page gives the relevant commands. For me it looks like gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0 (and repeat for ada1, ada2). And I guess that /boot/gptzfsboot file gets updated when I install a new kernel?
 
jwdevel said:
First, there is the question of how to arrange the ZFS datasets — the main ones being "shallow" or "deep", as described in the bectl(8) manpage.
Click to expand...
I'm not aware of any extended documentation of "deep boot environments" and their use.
For a deep dive into how one could use boot environments with respect to upgrades, I recommend to have a look at Allan Jude's ZFS Powered Magic Upgrades. I noticed that you didn't mention the -t flag as used in bectl activate -t <myBE>; that may very well be a useful option to consider. More info & links in my message here.
 
I use option C and I can vouch for the fact that most of the time, the process is smooth. I think the only issue I've had was with upgrading from one release to another. I ended up reinstalling FreeBSD from scratch as I couldn't seem to get it sorted.

I would prefer not having to reboot if at all possible. My process is a bit lengthier because I create a new BE for each possible update, ie. I separate out freebsd patches, kernel, and packages. If there is 1 of each type, then I need to reboot at least 3 times. The only saving grace for me is that my 'host' system is a barebones FreeBSD installation, so it has few packages and rarely needs updated. This is all handled automatically and I am notified of updates and that a reboot is necessary.

But yeah, I am contemplating changing my strategy. My jails are snapshotted and if there is an issue, I handle it at that point in time via snapshots.

If there were a means to also load a new kernel without rebooting, that would be nice to avoid reboots altogether.
 
tOsYZYny said:
If there were a means to also load a new kernel without rebooting, that would be nice to avoid reboots altogether.
Click to expand...
That is difficult. I personally think the best that can be done is upgrade (base os, which is kernel plus userland) in a new BE, force upgrade all packages into that new BE (likely chroot install/upgrade) then a single reboot into the completely new versions.

Upgrading a running kernel to new versions without a reboot is extremely difficult and I'm not aware of any OS actually doing that.
 
I looked at the original post and I think I can modify my update process. It looks straightforward.

Instead of me calling beadm create and activate, I will instead first notify myself via email or terminal message, chroot and directly apply updates. Then, I will notify myself to reboot in the morning, and voila. The actual upgrade would happen earlier and that would save me time in the morning to just execute the reboot.

I agree, it is redundant to create so many BE and sit in front of the computer while the updates are applied as opposed to letting it to the updates beforehand and notify me to reboot.

In order to avoid having the live BE and new BE becoming out-of-sync, I will notify myself first when an update is detected so I know not to make any changes.
 
You must log in or register to reply here.
Back
Top