pkg and Boot Environments

D

distancesprinter

The freebsd-update utility automatically creates Boot Environment snapshots using bectl if the system supports it. I am curious, does pkg upgrade have any similar capability?

I have used beadm and now bectl to create boot environments (then activate them, then update) for a long time, but now that freebsd-update automatically creates a BE snapshot, I'm starting to question why I would bother going through the trouble (just activate the last known good BE snapshot if something breaks, instead of going through the trouble of manually creating, naming, and activating). This seems more natural to me and, assuming I am properly grasping what is happening, avoids the need to reboot before starting an upgrade.

However, since I'm not familiar with any similar behavior with pkg upgrade, it would seem that I still need to go through the old routine. Why doesn't pkg upgrade create a boot environment in the same way that reebsd-update does?
 
One may have 100s to 1000s of packages, which may get updated at various times. If for each pkg update a new snapshot is created, you may have 1000s of snapshots. Managing them would become a bigger chore!
 
I always create and temporarily mount a boot environment before upgrading the packages at the mount point.

Then unmount, activate the environment and restart the OS.

Code: 
% bectl list -c creation
BE                     Active Mountpoint Space Created
n264621-09c20a293280-c -      -          4.35G 2023-08-10 02:12
n264621-09c20a293280-d -      -          598M  2023-08-14 13:12
n264621-09c20a293280-e -      -          5.03G 2023-08-16 10:53
n264811-d1d7a273707a-a -      -          1.19G 2023-08-17 04:56
n264868-edacf4b4824a-a -      -          407M  2023-08-19 03:54
n264998-d9fee1d02178-a -      -          402M  2023-08-24 21:07
n264998-d9fee1d02178-b -      -          573M  2023-08-25 15:31
n265053-315ee00fa961-a -      -          458M  2023-08-28 11:13
n265053-315ee00fa961-c -      -          891M  2023-08-31 16:36
n265053-315ee00fa961-d -      -          74.4M 2023-09-02 07:52
n265135-07bc20e4740d-a -      -          139M  2023-09-02 20:12
n265135-07bc20e4740d-b -      -          682M  2023-09-05 19:49
n265135-07bc20e4740d-c -      -          649M  2023-09-06 18:33
n265135-07bc20e4740d-d -      -          513M  2023-09-15 05:24
n265350-72d97e1dd9cc-a -      -          193M  2023-09-16 04:25
n265350-72d97e1dd9cc-b -      -          819M  2023-09-17 04:02
n265350-72d97e1dd9cc-c -      -          882M  2023-09-19 12:45
n265350-72d97e1dd9cc-d -      -          782M  2023-09-23 12:09
n265538-915af883221a-a -      -          256M  2023-09-26 16:30
n265538-915af883221a-b -      -          321M  2023-09-28 09:05
n265538-915af883221a-c -      -          368M  2023-09-29 23:34
n265538-915af883221a-d -      -          305M  2023-10-03 15:20
n265538-915af883221a-e -      -          154M  2023-10-04 13:08
n265538-915af883221a-f -      -          6.07G 2023-10-08 09:54
n265813-3ddd961db1ce-a -      -          868M  2023-10-09 04:49
n265830-fee14577d590-a -      -          665M  2023-10-09 13:39
n265830-fee14577d590-b -      -          911M  2023-10-11 06:29
n265830-fee14577d590-c -      -          802M  2023-10-14 17:48
n265830-fee14577d590-d -      -          588M  2023-10-18 01:27
n266088-3943698c6dbe-a -      -          603M  2023-10-21 02:47
n266088-3943698c6dbe-g -      -          614M  2023-10-30 16:09
n266088-3943698c6dbe-h -      -          354M  2023-11-06 23:42
n266088-3943698c6dbe-i -      -          42.9M 2023-11-09 05:15
n266317-f5b3e686292b-a -      -          147M  2023-11-09 20:10
n266317-f5b3e686292b-b -      -          587M  2023-11-10 15:38
n266317-f5b3e686292b-c -      -          1.24G 2023-11-13 09:26
n266415-257b04454818-c -      -          751M  2023-11-23 06:49
n266415-257b04454818-d -      -          402M  2023-11-29 18:38
n266415-257b04454818-e -      -          523M  2023-12-03 06:11
n266797-0a958aa16fed-a NR     /          566G  2023-12-04 05:57
%

Never so many that occasional destruction, one at a time, is a chore.
 
I don't know if it's practically possible, but if it is then it will require some sort of config about which datasets to include in the snapshot. The reason is because every system can have different dataset layouts.

For example, the installer defaults to putting /usr/local on its own dataset. But rolling back, you might only want to roll back the binaries, and not any config changes you may have made. (NB: it has always seemed to me that /etc and /usr/local/etc should be on a separate dataset from the rest of the base system for that reason). You might also decide to put /usr/local on the same dataset as / so that your base system and packages are consistent.

Anyway, I would say this is something that should be scripted first. Instead of calling pkg update directly, you write a script that creates a snapshot, then updates the packages. If enough people do that, and common patterns emerge, it can be rolled into a tool. Or you could make a tool that works for you, and share it for anyone else willing to follow your conventions. Call it lectl for local environment control :)
 
Another issue is that you may want to roll back only a specific package for whatever reason. So for example you update port A then B then C then D but now you want to roll back only B. This is not possible with filesystem snapshots. Unless you put each port in its own jail.
 
patmaddox said:
For example, the installer defaults to putting /usr/local on its own dataset.
Click to expand...
It doesn't? At least not on 12 and 13. Haven't done a clean install with 14 yet. With the default install /usr/local ends up in zroot/ROOT/default and thus becomes part of the boot environment.
 
I agree with SirDice on /usr/local Default install does not make an explicit dataset for /usr/local so it winds up directly under /usr which is partly required for "non-deep" "shallow" or normal boot environments.
If you do "bectl create -r" it may create a new dataset for /usr/local. I don't know this, just reading the manpage.

Getting back to pkg upgrade:
I typically create a new BE if I want to upgrade to the quarterly packages which is when you may get breakages (like a package failed to build so it gets deleted on you) so I can easily roll back. Once I've done the big upgrade and verified that everything I need works, I'll delete the BE I created.

If that becomes a feature on the pkg command in the future (like freebsd-update does now) it saves me a step but I need to remember to clean up the automatic ones.
 
It *should* default /usr/local to a separate dataset. Ideally *all* of port stuff should be on its own dataset including what is in /var/db/{pkg,ports}. So that you can even nfs mount /usr/local if you want. More and more I see the need for a plan9 style bind syscall!
 
Let's be pragmatic. ZFS snapshots are not the solution to all kind of problems with the upgrading of packages. They are great when making a snapshot just before you are doing a bulk pkg upgrade and you run into problems. With rolling back you get your working workplace fast.

But not all problems are so obvious because they break your all day tools. Packages do have dependencies and are dependent on.

What I miss is a tool that re-installs the last working version in a convenient way, including taking care of the version of the dependencies and reinstall them too if necessary.
 
Great discussion here--thanks for all the comments.

getopt said:
What I miss is a tool that re-installs the last working version in a convenient way, including taking care of the version of the dependencies and reinstall them too if necessary.
Click to expand...

Yes, that seems like it would be ideal and potentially even better.

For my purposes, I'm not sure it matters whether it's a BE or backup/revert packages, but the goal is simply to get the system back to it's working state immediately so that the admin can reason about what broke and why, without haste and anxiety about returning the system to a working state. With snapshots that seems easy to achieve. And I would argue for default on because it could save someone who forgot to take a snapshot before performing the pkg upgrade. Snapshots are cheap, it's nice insurance. Someone could turn it off if they were worried about destroying snapshots, or if they were doing very frequent pkg upgrades.

Regarding selectively rolling back packages, not sure that's ever going to be viable way to achieve returning the system to working state because of dependency hell.
 
Sorry to revive this thread, but I never came to a conclusion about how I should be using BEs.

It seems like there are two approaches--is one any better than the other?

1. If I never knew about Boot Environments, I'd have started out with a BE named default, and freebsd-update would simply create a clone of my currently running BE (default) with the name from freebsd-version -k and a datetime string. My updates would apply to default and I would get default upon next reboot and I'd be none the wiser. But if something went wrong, I could boot to a previous BE, the most recent of which was a clone of the currently running BE before freebsd-update made changes to default.

2. The alternative is the way I'd always used Boot Environments up until freebsd-update started creating them for me: I would create a BE, then activate it, then boot into it, then perform the update and continue on my merry way running the newly created BE with updates applied unless there was a problem. Coincidentally, this seems to be essentially the same as using the bectl mount feature and running freebsd-update by specifying the basedir and workdir of mounted BE (except that it saves the trouble of a reboot to boot into the newly created BE before running freebsd-update).

Question from my initial post was essentially:
> I'm starting to question why I would bother going through the trouble [of creating and activating/mounting BEs to do updates, as opposed to just creating them prior to doing the update, and keeping default as my running BE.]

How do you use the BEs? Is one of these methods better than the other? It seems like most articles about this advocate doing the latter, and I'm not sure why.

One more point: I think it's important to know how you're using the BEs when naming them. If you are using method #1, then the name should describe the state of the machine prior to an update, whereas if you're using method #2, then the name of the BE should describe the future state of the machine, since that will be the BE you are using after the updates are applied. And herein lies the source of my confusion when I encountered the new behavior of freebsd-update.
 
When I am upgrading across a release, say 13.x to 14.x, I'll manually create a new BE and then run freebsd-update and pkg commands using their chroot options. Why? Because I only have to reboot once and I'm in a consistent system. Basically I think that is similar to your #2. Again I only use when upgrading across to a new release.

Staying within a release say 14.x-p0 to 14.x-p1 I'll let freebsd-update handle the creation of new BEs. The important thing to remember is freebsd-update creates a new BE BEFORE doing anything.
You are currently in a BE named default.
You run freebsd-update install: that gives you a new BE named blahblahblah (representing the system before doing anything) and all updates wind up in the BE named default.

What I do is then bectl rename default to the new system, like 14.1-RELEASE-p2 to match the updated system.
So the BE starting out as "default" gets renamed after after freebsd-update.

I think the important point is to understand that freebsd-update creates a new BE representing the system BEFORE anything is done. I keep those around while I verify the system, after a while, I bectl destroy them.

The behavior of freebsd-update creating new BEs isn't all that new; I think it is actually a configurable and at some point the default changed. When, I'm not sure, but I'm pretty sure it's been "make a new BE by default" since at least 13.x
 
mer said:
What I do is then bectl rename default to the new system, like 14.1-RELEASE-p2 to match the updated system.
So the BE starting out as "default" gets renamed after after freebsd-update.
Click to expand...

It's funny that you mention this--I had never considered renaming a BE. Do you know if bectl rename also renames the underlying zfs snapshot?
 
I disabled the automatic generation of BE before an update/upgrade in /etc/freebsd-update.conf: CreateBootEnv no. Because the generated names are too long to type when time comes to delete them. I use very short names. In general, I only keep the two last BE.

For me the simpler is the better. I create a BE, make the update/upgrade or anything I have some doubts about and that's all. If I want to rollback, I activate the saved BE, reboot, delete the "default" BE and rename the saved BE "default". As a rollback is rare, this method is globally less tiring.

PS: I use almost only sysutils/beadm, as I have a very limited confidence concerning bectl(8).
 
Emrion I started out using beadm when ZFS first came into FreeBSD, but since bectl is part of base I've stopped using it. For my uses I've not had any issues, so why do you have limited confidence in bectl?

I've never liked the name "default"; I've always renamed it to reflect the output of freebsd-version because if I need to stop in the loader and select a BE I want to know what I'm booting into.
But threads like this are good because everyone has their way, sometimes we learn new tricks from each other.
 
You must log in or register to reply here.
Back
Top