Upgrade from 12.0 to 12.1 results in kernel panic

[Relictrix]

Hi,

When upgrading from 12.0 to 12.1, the upgrade itself does not return any errors. When rebooting the system, the startup with new kernel results in a kernel panic.

* What are the next steps to see in the logs ? I cannot startup so what's the procedure to read the logs?
* I tried to restart the old kernel but that results in going to the mountroot prompt.
* Why does the server end in the mountroot prompt?
* What does he need to know from the mountroot?

Mounting from zfs:interno/ROOT/default failed with error 5

Loader variables:
  vfs.root.mountfrom=zfs:interno/ROOT/default

Manual root filesystem specification:
  <fstype>:<device> [option]
      Mount <device> using filesystem <fstype> and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/cd0 ro

?                      List valid disk bootdevices
.                       Yield 1 second (for background tasks)
<empty line>  Abort manual input

mountroot>

Best Regards,

R

 

[tingo]

Have you tried to read the errata? (Hint: it is where you found the release notes)

 

[Relictrix]

Hi Tingo,

Good idea, I did not. Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.

For your information, as I thought perhaps the upgrade failed. I rolled back to snapshot and gave it another shot. Of course after doing the upgrade again, it's exactely the same error. I now doubt that it was error 5 or 45. But at the end I come back to mountroot. I must say I don't really know what's the purpose of mountroot. I read some other threads here related to all kind of mountroot issues. But some people then give statements to execute, but not why. That's why i decided to make a new thread. My experience with mountroot is low , very low. I never used it before . But the more trouble the more I learn. And I don't just want to try things on that machine, i still need that data.

 

[SirDice]

Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.

I'm going to bet it's this:

[2019-11-04] A late issue was discovered where systems using the graphics/drm-kmod port built on FreeBSD 12.0-RELEASE will crash during boot. It is advised to remove the port prior to upgrading, and build the port instead of using the upstream binary package.

 

[Relictrix]

Hi SirDice,

I will check that one soon , don't remember by heart if I installed it.

As your approach to finding this issue you really base yourself to the errata. I mean is there a way to find the error from the logs. I cannot open the 12.1 kernel, old kernel gives mountroot. Perhaps i could try to open the pool readonly and check some log. Or you got another, a better approach?

Best Regards,

R

 

[SirDice]

It's a common problem with users that have additional kernel modules installed. The packages in the repositories are still being built for 12.0. For 99% of the packages it's not a problem installing those on 12.1. Notable exceptions however are kernel modules, these will often cause the system to crash during boot.

This is one of the kernel modules that's been giving lots of users grief after they upgraded their systems. So it's my first guess it's been causing problems for you too.

 

[Relictrix]

Ok i see, I just noticed that i read the release notes not the errata. I want to go to 12.1, I will check the coming days and will respond back the result here. I already rolledback, will make a snapshot + backup externally. Then I will try again.

 

[SirDice]

Another common cause for crashes during boot after an upgrade (besides the aforementioned drm-kmod kernel module) are the VirtualBox kernel modules. So before you attempt the upgrade again make sure to disable any (third-party) kernel modules to avoid any potential issues. Kernel modules that come with the OS itself are not a problem.

 

[Relictrix]

Hi, the port graphics/drm-kmod was there, I deleted it, and run the upgrade. Will update here when the upgrade has finished.

 

[Relictrix]

Hi,

The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.

After restarting the pool in read only mode by starting the server in Single Boot mode via a USB disk, I could not see any files in /var/log or /var/crash. I wanted to search in detail for the error. Seems this got something to do with the mount, i mean normally there should be files there.

Just let me know if anybody has any thoughts, after a while I will probably rollback everything again because I need the server.

Best Regards,

R

 

[SirDice]

The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.

Do a pkg autoremove, graphics/drm-kmod is a so-called meta-port, it doesn't install anything of its own and pulls in one of the other drm-*-kmod ports/packages. And it's that dependency that actually has the kernel module.

Alternatively, just remove the kld_list from rc.conf. Or from loader.conf if you used that.

 

[Relictrix]

Hi,

I did that still does not work. Perhaps I have to be more drastic and should delete a lot more ports?

Best Regards,

R

 

[tingo]

First check that your zfs root pool is ok. Boot from a liveusb and check it.

 

[Relictrix]

My pool is ok. I did one more time, this time with

pkg delete -a

followed by a

pkg autoremove

. Result is still an error.

More details about the error are: fatal trap 12: page fault while in kernel mode.