Solved How to properly update a system

Previously, I hosed my system through an automatic update. Fortunately though, I was able to recover.

That said, I was perusing:

And, I am following all of the steps except:
Code:
pkg upgrade

It didn't say anything about any options we should use or avoid.

I think I was previously advised I shouldn't take all package updates because there could be random ones that could hose my system and that has happened to me at least once.

So, to make the process a bit more robust, I do:

Code:
1. check for updates
  freebsd-update fetch updatesready
  pkg upgrade -n (return code == 2)
  pkg audit -F (non-zero return code)

2. if updates are available
    a. determine latest patch sequence and increment it
    b. create new Boot Environment (including the patch)

3. instruct myself to reboot into the new BE
4. upon reboot (@reboot cron job)
     run freebsd-update fetch install
     run pkg upgrade -v -y (only install vulnerability fixes)
     check for further updates and notify the user

5. use the system as normal, if any problems are detected, revert to previous BE

I am wondering, but perhaps what I should do is once this is done, check if there are other updates available and create a separate BE for that so if the non-vulnerable updates broke the system, I can at least keep the vulnerability fixes. I'm confirming that it is a bit risky to just take all pkg updates regardless.

I would also like to have "the latest and greatest" to the extent that it is stable and reliable.
 
1643089695204.png
 
I hosed my system through an automatic update.
Blindly running updates is never a good idea.
I think I was previously advised I shouldn't take all package updates because there could be random ones that could hose my system and that has happened to me at least once.
I think you misunderstood or you are taking advice for a specific issue out of context.
 
I think you misunderstood or you are taking advice for a specific issue out of context.
That is entirely possible.

So, that leads me back to my original question, the PDF from the foundation didn't specify any arguments, so it would be updating everything; however, that doesn't sound like the best (wisest / safest) approach.

Would the best approach then be just to do:
Code:
pkg upgrade -v

So that only the vulnerabilities are patched and then if I feel lucky (after creating and booting into a new BE), do the remaining,
Code:
pkg upgrade
?
 
Best way is to run pkg upgrade and let it update/upgrade everything it wants too. Dependencies are set at build time when the package is created. If you upgrade PackageA but not PackageB you can get discrepancies with this dependency chain. The newer version of PackageA might be linked to a newer version of PackageB. If you only updated PackageA then it can fail because PackageB is still at the old version.

Treat all the installed software as a whole, not as individual packages. And keep an eye on /usr/ports/UPDATING before updating anything.
 
About the only option I ever use on pkg upgrade is "-n" Why? So I can see what it wants to update and think on it. If I currently have firefox open doing something for paying work, I don't want to pkg upgrade firefox. It also lets me have a chance to see what depends on what, a chance to decide "do I need that". Otherwise, I just let it run.
I'll sometimes create a new BE to upgrade into but not always, because BEs can accumulate like old newspapers.
 
Best way is to run pkg upgrade and let it update/upgrade everything it wants too. Dependencies are set at build time when the package is created. If you upgrade PackageA but not PackageB you can get discrepancies with this dependency chain. The newer version of PackageA might be linked to a newer version of PackageB. If you only updated PackageA then it can fail because PackageB is still at the old version.

Okay, that makes sense. It would be good if that were explained somewhere. I would think that since the pkg command has the -v (vulnerabilities) flag that it would imply it is smart enough not to hose your system and that if pkg A depends on pkc C, that it will also pull in pkg C.

In regards to BE's, yes, that is something I'm looking to script with a cron job or find an existing pkg or tool that does it. I want to periodically remove stale BE's whilst leaving me an escape hatch. I really like how zap works, so I am thinking that I will use that as a reference and write some shell scripts with cron to achieve a similar thing with BE's.
 
About the only option I ever use on pkg upgrade is "-n" Why? So I can see what it wants to update and think on it.
pkg gives you by default first a summary and asks you to proceed. So I don't use this option.
 
  • Like
Reactions: mer
Okay, that makes sense. It would be good if that were explained somewhere. I would think that since the pkg command has the -v (vulnerabilities) flag that it would imply it is smart enough not to hose your system and that if pkg A depends on pkc C, that it will also pull in pkg C.
During the three month period of a quarterly branch you can probably safely do pkg upgrade -v. During that three month period updates will be far and few in between. And will only have security and/or major break fixes. So there can't be any significant version differences. Just after a change-over (from Q1 to Q2 for example) you need to be more careful though. You will get all the changes and updates that happened in that three month period on latest all at once. This can include potentially breaking updates, like a change from PostgreSQL 12 to 13. Or the next default version of Ruby, Perl, Python, LUA, etc.

The less often you update the bigger the jumps are going to get and therefor the bigger the risk of breaking changes. So my mantra is really, update regularly. If you track quarterly (-RELEASE versions do by default) at the very least once every three months.
 
Code:
…

2. if updates are available
    a. determine latest patch sequence and increment it
    b. create new Boot Environment (including the patch)

3. instruct myself to reboot into the new BE

…

If the installation of updates at step 2a creates an environment that is problematic with problems not immediately obvious, then what earlier boot environment do you have?
 
If the installation of updates at step 2a creates an environment that is problematic with problems not immediately obvious, then what earlier boot environment do you have?

The script / cron I plan to create is that I would expire old BE's after a period of 30 days or some sort of timeout. Once the BE is both not current and not scheduled to be active for the next reboot and is > 30 days since it was created, then I will destroy it.

I expect to catch problems when the patches are applied +- a day should let me touch all of the stuff I use on a day-to-day basis. Anything beyond that would fall into the bucket of general triage / break / fix.
 
I mean, it seems that you install updates before creating a new boot environment. To me this seems risky, because you might subsequently find problems with the updated pre-new environment.
 
I don't believe so, maybe I misstated something.

1. check if updates are available, if yes, create new BE, and prompt user to reboot into BE
2. upon booting into new BE, utilizing cron via @reboot schedule, apply updates, repeat #1, if the user has not booted into the new BE (I create a patch file indicating the BE and what updates are to be applied), then a message will be logged and no updates applied.
3. attempt to prune old BE's daily


I tested out 1 and 2 today and it works, time will tell what I need to fix. I will test the pruning process tomorrow. Something like this needs to exist just like the beadm utility.
 
Don't believe everything you hear.
Not everyone who acts like they know what they're talking about actually does.

I would also like to have "the latest and greatest" to the extent that it is stable and reliable.
Me, too. That's never involved booting into a new BE in 16 years.

What it does involve is updating the ports tree snapshot, checking for new vulnerabilities and running freebsd-update on a daily basis:

Code:
portsnap fetch update
pkg audit -F
freebsd-update fetch

When there's a vulnerability in a port I upgrade as soon as it's available in the ports tree using ports-mgmt/portmaster.
Whether or not I used pkg to install it, but that's me.

When there is an update to freebsd-update I run it as soon as it's available and then reboot:

Code:
freebsd-update iinstall
shutdown -r now

I run portmaster -a when I want to update my 3rd party rograms because ports are updated more frequently than pkg.
 
Thanks, now I can visualise that part.

… Something like this needs to exist just like the beadm utility.

vermaden can you think of anything?

I don't have a link handy, but I think first of code that might be archived in GitHub for PC-BSD.



On the GhostBSD side, FYI Better use of boot environments with Update Station - GhostBSD, the key points are probably:
  • currently, Update Station creates a new boot environment as a backup
  • grahamperrin could you make a Feature Request about this?
  • … most of the major OSs upgrade on boot and not on the desktop. … will be needed for 13 upgrades to 14.

Also, less directly relevant: <https://github.com/ghostbsd/software-station>



On the FreeBSD side, re: the Foundation's Technology Roadmap, I have done nothing recent with PkgBase. Awaiting feedback on a bug report.

 
vermaden can you think of anything?

I do not think its that easy.

When less downtime is needed then I prefer this way of upgrade:
https://vermaden.wordpress.com/2021/02/23/upgrade-freebsd-with-zfs-boot-environments/

I do not think its possible to safely automate pkg upgrade or freebsd-update install parts. Boot Environments will just create a 'safe' BE that you can get back to safely.

You can not automatically 'just do' pkg upgrade -y because in the process of SAT solver (or any other automatic solver) you may end up with some packages removed. That means that for example net/samba will be removed as a conflicted dependency and you suddenly do not serve your SMB shares now ...

You can not automatically 'just do' freebsd-update install because sometimes config files need to be merged. You can end up with just plain 'skel files' that are part of fresh FreeBSD installation or with your config but without new group or new service user that would break some part of the FreeBSD Base System.

For the record I do not think that PKG Base would solve that problem either - for the same reasons I gave for pkg upgrade command.

I may be wrong of course (and often am) but with current way how the pkg(8) and freebsd-update(8) work I do not see the point in such automatic updates. Of course the created BE would keep you safe and you can reboot back to working BE that you created before that automatic update process.

Also its not always needed to update. For example new security hole was found in Bluetooth stack. If you do not use Bluetooth at all you should just read the SA to check if its really harmless to you. Another example may be a bug in IPSec Tunnel implementation. If you do not use it then read SA and make sure upgrade is not needed. Same with packages. IMHO that takes less time (and less downtime) then having to reboot(8) bunch of servers into working BE created before automatic update - then disable the auto-update-script.sh thing temporary - then fix it (or make workarounds if not possible to fix) - then keep fingers crossed that this will not break again.

My $0.02.

Regards.
 
Another thing ... if the kernel is not involved in the update why reboot?

Create new BE.

Update currently running BE.

Then use sysutils/checkrestart to see what needs to be restarted.

Restart it.

Continue running without reboot.

Example output from sysutils/checkrestart shown below (from my laptop/desktop system).

Code:
# checkrestart
  PID   JID USER         COMMAND      WHY ARGUMENTS
81541     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
93327     0 vermaden     geany        .so /usr/local/bin/geany
56719     0 vermaden     telegram-des .so /usr/local/bin/telegram-desktop
81939     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
49428     0 vermaden     gvfs-udisks2 .so /usr/local/libexec/gvfs-udisks2-volume-monitor
56988     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
81820     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
74021     0 vermaden     firefox      bin firefox
31141     0 vermaden     openbox      .so /usr/local/bin/openbox
48933     0 vermaden     dzen2        .so /usr/local/bin/dzen2
78890     0 root         urxvt        .so /usr/local/bin/urxvt
65326     0 vermaden     deadbeef     .so /usr/local/bin/deadbeef
41393     0 vermaden     mkvtoolnix-g .so /usr/local/bin/mkvtoolnix-gui
78391     0 vermaden     urxvt        .so /usr/local/bin/urxvt
85055     0 vermaden     soffice.bin  .so /usr/local/lib/libreoffice/program/soffice.bin
75328     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
82244     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
41422     0 vermaden     tint2        .so /usr/local/bin/tint2
76115     0 root         Xorg         .so /usr/local/bin/Xorg
75228     0 vermaden     dbus-daemon  .so /usr/local/bin/dbus-daemon
66143     0 vermaden     caja         .so /usr/local/bin/caja
 7014     0 vermaden     pulseaudio   .so /usr/local/bin/pulseaudio
81274     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
32637     0 vermaden     transmission .so /usr/local/bin/transmission-gtk
 
S/he will be disappointed. Due to the laws of thermodynamics, people can't be a power source, it requires more energy to keep them alive than they can produce. Matrix is just a variation on perpetuum mobile, completely impossible :p
 
That wasn't the blue pill I was going to take.

Another thing ... if the kernel is not involved in the update why reboot?

It is my understanding that a new BE requires a reboot. If the update is problematic and I don't create a new BE just because it doesn't have kernel updates, then some user application will be impacted.


Example output from sysutils/checkrestart shown below (from my laptop/desktop system).

Another good point, once done, I should record that and notify the members of the wheel group.

I think the bottom line is that these updates cannot be 100% automated because they might involve configuration changes . In that case, it'd still be nice to automate as much as possible and guide the user through the process just like the freebsd installer does at install time.
 
Back
Top