Solved Endless Reboot after Update 15.0 to 15.1, bricked on reboot after rollback

K

kennethcf

I just had a catastrophic failure of the worst kind and am not quite sure what I should do.
Any help would be appreciated.

I just upgraded from 15.0 to 15.1 after applying latest security patches and rebooting.
# freebsd-update -r 15.1-RELEASE upgrade
It got stuck in a reboot cycle. Last I remember were lots of drm error messages, then reboot.
On one of the reboot cycles, I selected the previous kernel, which worked.
I executed a rollback.
# freebsd-update rollback
I got no errors, so I shutdown -r now
That was the last I saw of my system.
After rebooting, the console eventually stopped on
Loader variables:
vfs.root.mountfrom=zfs:zroot/ROOT/default
Manual root filesystem specification:
<fstype>:<device> [options]
Mount <device> using filesystem <fstype>
and with the specified (optional) option list.
e.g. ufs:/dev/da0s1a
zfs:zroot/ROOT/default
cd9660:/dev/cd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
? List valid disk boot devices
. Yield 1 second (for background tasks)
<empty line> Abort manual input
mountroot>

UPDATE
Apparently, the above is the result of the "freebsd-update rollback" not rolling back the kernel. It still boots into the new kernel.
I hit return to "Abort manual input" and after another message about not finding a partition, it finally restarted and I could select the old kernel. I guess I am not bricked. But I do need to remove the new kernel.

Anyone have a link to a post on removing a new kernel from the boot options and restoring the old kernel as the default?

I think I'm going to have to go through this again if I am to find out what is going on with drm and why the reboots. Am I forever stuck on 15.0?
 
A 15.1 kernel crashes without panic message? I had that with 15.1RC2 but it doesn't happen with the finished 15.1.
But what computer is this? If it supports USB boot, you can check if the installer kernel works to make sure that's the problem. You can also change things on an installed system with the live shell, like faulty /etc/fstab settings.
 
Thanks MG. I don't know if it actually panicked, or it just rebooted.

It is an AOOStar Gem12 Mini AMD Ryzen9 6900HX w/internal Radeon 680M. After being replaced for not turning on, it has been stable. Yes, it supports USB boot. I had to install drm-515-kmod to get the graphics to work on 14.1 and had to change to drm-66-kmod when I moved from FreeBSD 14.1 to 15.0. See https://forums.freebsd.org/threads/...n-cpu-and-gpu-7-8-9-family.85401/#post-752347

Additional update - Things are not exactly perfect after a rollback and booting into kernel.old. I don't know if I need to be worried or do an update of security patches again and all will be well.
freebsd-version -k --> 15.0-RELEASE-p6
freebsd-version -u --> 15.0-RELEASE-p6
uname -r --> 15.0-RELEASE-p10

I noted problems with drm on other posts with 15.0 STABLE a couple of months back and I wonder if this is related or similar. Maybe 15.1 requires drm-66-kmod to be compiled on a 15.1 system from source before it works? I'll work on this in the morning.
 
kennethcf said:
Maybe 15.1 requires drm-66-kmod to be compiled on a 15.1 system from source before it works?
Click to expand...
I guess that's all your problem. There is a simpler method than compile: upgrade you kmods repo.
pkg upgrade -r FreeBSD-ports-kmods. But you need to have both kernel and userland on 15.1 for the upgrade to work.

The process is:
- Deactivate all third-party kernel modules loadings at poweron, including drm.
- Upgrade the kernel (with freebsd-update).
- Reboot.
- Upgrade the userland (with freebsd-update).
- Upgrade the packages that belong to FreeBSD-ports-kmods repos.
- Check if the kmods are really upgraded (you must see 1501 as part of the package name and not 1500).
- Reenable the kmods loadings.
- Reboot.

Note: IIRC, freebsd-update asks for the deletion of old libs for 15.1. So, run a last freebsd-update install.
 
I've never needed:

vfs.root.mountfrom=zfs:zroot/ROOT/default

so I don't know exactly what it does. It's obvious that it prevents activation from bectl, but it's not obvious that it breaks the manual selection of a BE from the boot menu. Have you ruled out selecting freebsd-update's backup BE in that way?
 
Update: 15.0-RELEASE has been restored. After booting kernel.old, I renamed /boot/kernel to /boot/kernel_failed_15.1 and /boot/kernel.old to /boot/kernel, so it would default boot to the good kernel yet keep the old one for forensic purposes. Next, after booting again into the default kernel (the working kernel), I ran "freebsd-update fetch install" which corrected the 3 environments. "freebsd-version -kru" now shows 15.0-RELEASE-p10, instead of a mix of 15.0-RELEASE-p6 for installed kernel and userland and 15.0-RELEASE-p10 for the running kernel. (DISCLAIMER - I used ChatGPT).

Next: Diagnose what went wrong.

Update:

All is restored. Thank you Emrion for you insight.
I paid attention to the warning from "freebsd-update -r 15.1-RELEASE upgrade" about all 3rd party packages that needed to be rebuilt, and stopped the upgrade.
Instead, I turned off amdgpu by commenting it out in /etc/rc.conf
#kld_list="amdgpu" <-- commented out
Then I removed all the kmod packages".
# pkg remove -g gpu-firmware*
I also removed drm-66-kmod-6.6.25.1500068_8
# pkg remove -g drm-66-kmod-6.6.25.1500068_8
I rebooted and my graphics were gone, as expected.

I proceeded with the upgrade, "freebsd-update -r 15.1-RELEASE upgrade", and EVERYTHING went smoothly!

# freebsd-version -kru now says 15.1-RELEASE for all 3 after the twice "freebsd-update install" install cycles for userland.

I upgraded pkg.
# pkg upgrade pkg

I already had the ports tree but cloned src releng/15.1 fresh, cause you need src to build the kmod driver. I didn't realized I had a very old ports tree, 2025Q1. I built the flavor of graphics/gpu-firmware-amd-kmod I needed. (It was a long process of removing each driver and then seeing if the graphics still worked to find out my FLAVOR was yellow-carp.)
# make FLAVOR=yellow-carp install clean
and a package for my jails
# make FLAVOR=yellow-carp package

When I saw the old date in the yello-carp package name, I decided to take a chance and just "pkg upgrade". My yellow-carp upgraded!
I installed drm-66-kmod, and it said it was specifically built for 15.1-RELEASE -- that was good.
# pkg install drm-66-kmod

After turning on my amdgpu in /etc/rc.conf and a final reboot, X and everything is working again on 15.1-RELEASE !!!
 
You must log in or register to reply here.
Back
Top