12.2 userland on 13.0 kernel?

[zirias@]

Quick question: Should a 12.2 userland run correctly on a 13.0 kernel, or should I expect problems (like e.g. with bhyve)?

Background: I'd like to only upgrade the kernel first and rebuild my package repo with poudriere using a 13.0 jail before completely upgrading the host…

 

[SirDice]

Should a 12.2 userland run correctly on a 13.0 kernel

It should, it's supported as a jail. That's what the COMPAT_FREEBSD?? kernel compatibility options are for.

I'd like to only upgrade the kernel first and rebuild my package repo with poudriere using a 13.0 jail before completely upgrading the host…

If you upgrade with freebsd-update(8) run the freebsd-update install twice. That will update the kernel (first run) and the userland (second run). At this point your 'old' 12.2 binaries should still work. Now do all the rebuilds. When you've updated all your packages to 13.0 equivalents ( pkg upgrade should detect the change) you can run the last freebsd-update install. That last run will remove all the old libraries for 12.2 that allowed you to run the 12.x binaries without problems.

If you do a source upgrade, run the usual installkernel, installworld and mergemaster(8). Then rebuild your stuff. A make -DBATCH_DELETE_OLD_FILES delete-old and make -DBATCH_DELETE_OLD_FILES delete-old-libs will remove any old 12.x stuff afterwards.

This is what you're going to see:

root@riviera:~ # freebsd-update install
src component not installed, skipped
Installing updates...
Kernel updates have been installed.  Please reboot and run
"/usr/sbin/freebsd-update install" again to finish installing updates.

I usually don't even reboot at this point.

root@riviera:~ # freebsd-update install
src component not installed, skipped
Installing updates...sysctl: unknown oid 'user.localbase'
Scanning //usr/share/certs/blacklisted for certificates...
Scanning //etc/ssl/blacklisted for certificates...
Scanning //usr/share/certs/trusted for certificates...
Scanning //etc/ssl/certs for certificates...

Completing this upgrade requires removing old shared object files.
Please rebuild all installed 3rd party software (e.g., programs
installed from the ports tree) and then run "/usr/sbin/freebsd-update install"
again to finish installing updates.

This step updated the userland. Note the message about rebuilding things. Now is a good time to reboot and start rebuilding your ports.

root@riviera:~ # freebsd-update install
src component not installed, skipped
Installing updates... done.

This is the third and last step. This will remove all the old libraries.

If you need to run 12.2 binaries at this point (because you haven't rebuild everything yet for example) you can install misc/compat12x to install the 12.x libraries.

If you're on ZFS don't upgrade the pool right away, make sure everything is in order first. If you upgrade the pool you can't go back to 12.2 any more. So this is the absolute last step you should take, if and when you're absolutely sure everything is in order.

 

[zirias@]

Well, thanks for an elaborate answer! Unfortunately, it's not that simple.

Meanwhile, I just tried, and, well, the immediate answer is: NO. Whole story:

What I found broken was networking (e.g. promicious mode couldn't be set on any interface, breaking all of my bridging) and bhyve (machines booting with uefi couldn't start any more). There's probably more. So, sure, a jail with a 12.2 userland will run fine, but the main userland is a different story

This is a machine with lots of jails and vms doing everything for me at home. I don't want it down for too long, and I definitely can't risk having it broken.

So, as I need a 13.0 userland at least on the host, I tried to install it in a separate boot environment (to have a way back). make installworld also installs a few bits in /boot, and with a separate bootpool, this is "global" for all boot environments. But even worse, I found the loader won't boot my environment, which is probably also because of this:

Solved - [GELI] getting rid of bootpool?

I installed my server with 11.0-RC1, and back then, the installer created a separate (unencrypted) bootpool for ZFS root on GELI. Now, I know this isn't needed any more for a long time. Are there any instructions around how to migrate an existing installation to get rid of the bootpool? Anything...

forums.freebsd.org

Well, disaster hit me here, cause now, VMs using bhyveload also didn't boot any more. One of them is my central router/firewall, so, LAN and internet defunct. It took me a while to realize this must have been broken by the 13 make installworld installing stuff in /boot, and yet ANOTHER while to remember I had snapshots from my backups that also included my src and obj trees, so null-mounting THEM as /usr/src and /usr/obj, I could finally recover with another make installworld.

I guess I'll first have to solve that problem with the dreaded bootpool and get boot environments to work before starting another attempt.

 

[SirDice]

Maybe set up a 13.0 bhyve guest to build the packages you need on the host? So you can upgrade the host itself completely, the packages are already built for it. Once that's running you can use a 13.0 jail to build the rest of the packages you need. I assume you don't have that many packages installed on the host, so a small 13.0 bhyve guest could build those in advance.

 

[zirias@]

Yes, thanks, I thought of that (I already have a "testbuilder" VM which followed 13-CURRENT all the time).

My problem now is a different one: With my initial plan, running the whole machine JUST with the 13.0 kernel (and switching back to kernel.old is always easy), I hoped to gain confidence 13.0 will run fine on that machine. Now that this doesn't work and boot environments don't work either, I have to find a way…

Still this is a good idea I'll probably follow to minimize downtime. Just unsure yet whether I'll dive into 13 without a solid rollback plan, probably not. If you'd have any advice how to safely get rid of the bootpool (see linked thread, so, boot environments would hopefully work correctly), that would be great!