FreeBSD 13.2-RELEASE -> 14.0-RELEASE Upgrade Stuck?

I'd completely skip 13.2 as a ZFS user and just go from 13.1 to 13.3 (or soon 13.4). Though I think it got retroactively patched, 13 in general was getting progressively worse in my experience until 13.2 became nearly unusable with arc_prune calls and other performance bugs; on the flip side, I started reviewing my old performance metrics and management to find various issues and changes.

If you want intermediate steps, patch your current version, then upgrade it to the highest minor version number for your version, then push forward to the next version (normally straight to the highest minor version until documentation tells you not to). It is also wise to manually download a newer freebsd-update from a newer freebsd version before upgrading, which is normally the goal of many steps of "freebsd-update system to this version before further versions."
 
Alternatively, upgrading by building from source also avoids any "freebsd-update to the next major version is taking hours(s, even on an SSD?)", where even on slow magnetic drives it seems to install at a fine pace. Unfortunately that does require a lot of build time, but your system can stay in a ready to use state during that as long as CPU, RAM, and I/O doesn't get too loaded up with build tasks.
 
Alright so I went from 13.1 to 13.3. Took about an hour. So that’s reasonable. Let’s hope going straight to 14.1 is the same….

I literally hammered my FreeBSD 13 and it’s still alive, just wow on its redundancy.

I recently learned about “bectl”, it is your best friend for recovery. Crazy but I opted to upgrade and not fresh reinstall, my OS KDE environmental is highly customized and took 3 years in the making, don’t want all that hard work to disappear and redo again.

IMG_6088 Medium.png


IMG_6089 Medium.png


Also on the side not about the “not a directory” error, I also got that while upgrading from 13.1 to 13.3:

IMG_6087 Medium.png


Alright 50 mins passed by… and…
14.1 :):

IMG_6093 Medium.png


Seems like I have to clean all the mess 13 left over…:

IMG_6091 Large.jpeg


So if anyone curious…

It took 50 mins for 13.1 to upgrade to 13.3 and another 50 from 13.3 to 14.1. About 2 hrs take it or leave it.
I’ve learned a lot, crashed and hammered the kernel, got a basic real understanding now how ZFS File system functions and how to recover and deal with it, able to use bectl create ZFS snapshots to do quick emergency boot recovery points. Completely amazing.

I’m just shocked how fast the recovery process is. PC loads in 3 seconds, in 2 seconds I can choose which snapshot I want, and in 2 seconds FreeBSD can tell me what’s going on with the boot for the chosen snapshot.

It takes only 1 second to create a snapshot using bectl. Creating a snapshot, activating it and changing between snapshots to be the default boot ZFS drive is dead simple.

FreeBSD is inherently irresistible and resilient. I’m completely astound. I had no hopes to be able to upgrade to 14.1 (specially with over thousands of packages installed during my first tryout with FreeBSD over the years)… but I just wanted to test how good FreeBSD’s redundancy is so I gave it some more thought and it’s simply bulletproof. This FreeBSD copy will now be 8 years old (3 years from 13.1 and another 5 years while on 14.x until EOL and 15 comes out)
 
I have been experiencing this on UFS for some time when using
freebsd-update install, but I thought it was related to VM hypervisor bugs. I just ran into it again on a VPS that seems to be RedHat/KVM based, with an IDE(no NCQ) controller and 1 core.

That led me to look into the problem a little more, and discover these reports of similar effects on ZFS. (hours slow instead of hanging)

I am seeing /usr/bin/install go into D-state with a wdrain and never come back. Disk activity and CPU is 0 when this happens. I usually do with a hard reset because its a local instance, but for the VPS reboot -lqn worked.

I think there is some kind of deadlock/race condition in the kernel that is affecting UFS and ZFS differently, as I notice that install -S uses the rename() call.

Hopefully this helps someone narrow down where the problem could be or discover a way to reproduce it.
 
vfs-locking said:
I just ran into it again on a VPS that seems to be RedHat/KVM based, with an IDE(no NCQ) controller and 1 core.
Click to expand...
So there is very likely an extreme disk I/O bottleneck in place, especially if that VM runs on spinning rust.
During upgrade there are thousands of files being changed/replaced, so IOPS are crucial. The CPU going into idle is to be expected as you completely choked the disk and the CPU is always waiting for IO to finish.
 
You must log in or register to reply here.
Back
Top