Multi step major upgrade - reinstalling packages?

[et.fz]

Hi,

I'm going to upgrade from 12.4 via 13.5 to 15.1. I need to reinstall all packages after a major upgrade, but can it wait until 15.1, or is that going to cause issues somewhere along the way?

 

[SirDice]

I'm going to upgrade from 12.4 via 13.5 to 15.1.

I would recommend a clean reinstall. Too many things that could go wrong (unless you know what you're doing).

 

[drsnx60]

Why dont you move your data to backup storage first and install 15.1 clean. Or buy a second diskdrive to install 15.1 too.

 

[cracauer@]

I don't bother with either reinstall nor with reinstalling all packages. Works fine most of the time, but see my thread about my screwed up g+.

 

[Maturin]

Yeah, I also would recommend a clean new install.
As drsnx60 said, backup your stuff first, perhaps do a full backup, and then do a new installation, maybe on a fresh drive.
A new installation will take you less than an hour (not counting backing up your data, which you have to do anyway) while those many upgrade steps will take you days. And a zillion things can go wrong while this stunt, since several things changed since 12.4, like default shell changes from csh to sh - overseeing such thing, and you also have a mess to clean up, too.

 

[Charlie_]

For major updates, I reinstall about ten packages - including kmod.
As for the rest, I've installed misc/compat1?x and left it as is.
I am currently on version 15.1, but I still have dozens of 13.x packages as well as some 11.x and 12.x packages remaining.

 

[ralphbsz]

I would recommend a clean reinstall. Too many things that could go wrong (unless you know what you're doing).

I very much disagree with this as a general advice.

But I partly agree. For a machine with lots of software installed (typically a desktop), very few customizations (no special server-style packages with extensive customization and configuration), and files that are just in user directories and can be backed up and restored, I would agree with re-installing. In such a situation, users should actually move to storing their data in the cloud, and using browser-based apps, and treat their desktop machine as a disposable and stateless device.

On the other hand, for a machine that has complex customization, non-standard software installed, and that other things rely on staying up as much as possible, the correct answer is to upgrade seamlessly in place. And FreeBSD's strength is that upgrades are possible and usually relatively painless and safe, even across major versions. That capability needs to be exercised and maintained, both by individual admins and by the project.

Which category the OP's machine falls in: is up to them to decide.

 

[et.fz]

Alright, I appreciate that you're all my seniors here, but that is also part of the crux. This is an inherited web server, and while it's not a particularly complex setup, I don't know which minor customizations might be in place. I am not doing a fresh install unless there is an extremely good reason to do so. Why is upgrading even possible? You're making it sound like the feature is useless. If I had upgraded to each major as they were released, would that have been any better? If yes, why?

Yeah, I also would recommend a clean new install.
As drsnx60 said, backup your stuff first, perhaps do a full backup, and then do a new installation, maybe on a fresh drive.
A new installation will take you less than an hour (not counting backing up your data, which you have to do anyway) while those many upgrade steps will take you days. And a zillion things can go wrong while this stunt, since several things changed since 12.4, like default shell changes from csh to sh - overseeing such thing, and you also have a mess to clean up, too.

I don't see why it should take days? It's two steps. (versions) Waiting a little while for packages to download, and spend a couple of minutes merging some configuration files. Couple of reboots. The data is already backed up automatically. I have already tested the upgrade steps on a cloned machine, and it seems to work fine afterwards. I am changing my shell back to csh.

I didn't realise reinstalling packages was optional. I think that it breaks PHP packages, at least? I will just reinstall all packages. I would just like to know whether I need to perform this step for the intermediate version 13, but it's sounding like the answer is no, then.

 

[tingo]

Upgrades work fine - they are just time consuming. The upgrade itself is not the step that takes the most time (depending on if you do it in multiple steps or not).
But the reinstalling packages step - sounds simple right?

(before you do the upgrade, be smart and get yourself a list of all currently installed packages and puit it somewhere you can access it during the upgrade / reinstall.)

Well, here is what you have to deal with:
- running pkg upgrade multiple times, due to the fact that package dependencies have change over a few years
- running pkg install for the packages that didn't get upgraded
- finding replacements for the packages that have been removed from the collection

this takes a while. But the real timesucker is this one:
- fixing all necessary configuration changes (example: apache config files - format is the same, but which directives you need to have in there to get a config to work changes with at least some major versions)
- php - apache used to have a built in php modul, now you need to run it via fcgi or similar.
- database upgrades; if you use Mysql, Mariadb, or PostgreSQL, get ready to spend some quality time with all official and unoffical documentation, upgrading from older version (say 4.x, 5.x to 8.x) is never quick or easy.

there is probably more, but I'm getting tired just by writing this.

 

[mer]

This is a big jump, I think if you use "single user mode with networking" you can mitigate the number of pkg upgrades.
Folk normally capture the output of pkg prime-list to get the list of installed packages which makes it easier to reinstall.
But: you are going from 12 to 15 and in that time I'd guess things got deprecated, things like python. So when you get to 15 you may wind up pkg install python version from 12 which you probably don't want.

Did you install 12 on ZFS? If so you may be able to get away with explicit upgrade into a new BE which you can do without a reboot until the end.

All the comments about configuration are key so pay attention.
Do you have space/ability/resources to install a new blank device that you could directly install 15 on, then work on making that coherent with the 12 system? I've used that method a lot when I am making a big jump (I consider 12 to 15 a big jump)

 

[bob2112]

Upgrades work fine - they are just time consuming. The upgrade itself is not the step that takes the most time (depending on if you do it in multiple steps or not).
But the reinstalling packages step - sounds simple right?

(before you do the upgrade, be smart and get yourself a list of all currently installed packages and puit it somewhere you can access it during the upgrade / reinstall.)

Well, here is what you have to deal with:
- running pkg upgrade multiple times, due to the fact that package dependencies have change over a few years
- running pkg install for the packages that didn't get upgraded
- finding replacements for the packages that have been removed from the collection

this takes a while. But the real timesucker is this one:
- fixing all necessary configuration changes (example: apache config files - format is the same, but which directives you need to have in there to get a config to work changes with at least some major versions)
- php - apache used to have a built in php modul, now you need to run it via fcgi or similar.
- database upgrades; if you use Mysql, Mariadb, or PostgreSQL, get ready to spend some quality time with all official and unoffical documentation, upgrading from older version (say 4.x, 5.x to 8.x) is never quick or easy.

there is probably more, but I'm getting tired just by writing this.

For most of this doing a series of reinstalls would be worse.

 

[ralphbsz]

Alright, I appreciate that you're all my seniors here, but that is also part of the crux. This is an inherited web server, and while it's not a particularly complex setup, I don't know which minor customizations might be in place.

In that case, upgrades are LIKELY to be less painful than a reinstall. Because if you do a reinstall, you need to EXACTLY KNOW which minor customizations are place.

Why is upgrading even possible? You're making it sound like the feature is useless.

Upgrading is extremely possible. As I mentioned on this forum before: I've continuously used FreeBSD on my server machine for about 12 of 15 years. I've only re-installed twice. Once, because I was lazy and had skipped 3 or 4 major versions, and at that point upgrading was too tedious. The second time because 32-bit x86 mode was no longer supported, and I had to switch to 64-bit mode, which requires a reinstall.

If I had upgraded to each major as they were released, would that have been any better? If yes, why?

It would have been exactly the same as what you are going to start now, except the total amount of work would have been distributed over many years. Actually, let me take that back: Doing all the upgrades at once is more efficient, because you get into a rhythm, and only need to do serious testing once at the end. You can also skip upgrading / re-installing packages until the very end, and that's typically where the trouble happens (the base OS is incredibly reliably to upgrade).

and spend a couple of minutes merging some configuration files.

See below ... sometimes "merging some configuration files" is not as easy as you think,

I have already tested the upgrade steps on a cloned machine, and it seems to work fine afterwards.

In that case, you're winning!

I didn't realise reinstalling packages was optional. ... I would just like to know whether I need to perform this step for the intermediate version 13, but it's sounding like the answer is no, then.

Don't need to. I wouldn't waste my time on it.

(before you do the upgrade, be smart and get yourself a list of all currently installed packages and puit it somewhere you can access it during the upgrade / reinstall.)

ABSOLUTELY, and with version numbers. Not just "python and a lot of little python things", but an EXACT list of each package name and version. I've had cases where packages change name!

- fixing all necessary configuration changes (example: apache config files - format is the same, but which directives you need to have in there to get a config to work changes with at least some major versions)

That is exactly the example I was going to give! It bit me this week. I have two FreeBSD servers, and because I'm super busy, they are right now different versions (I know I know, need to work on it). One Apache still uses "Order Deny,Allow; Allow from all"; the other now needs to use "Require all granted". The syntax for using environment variables in Allow/Require clauses has completely changed. I spent half a day until I had it back to work, what a pain. In particular if you didn't set up the machine, or if it was so long ago that you don't remember why it was set up that way, these kinds of things can be time sinks.

BUT: This kind of problem will get you exactly as badly if you reinstall, and hope to just copy your config files over.