Solved Upgrade from 12.0 to 12.1 results in kernel panic

Relictrix · Jan 13, 2020

Hi,

When upgrading from 12.0 to 12.1, the upgrade itself does not return any errors. When rebooting the system, the startup with new kernel results in a kernel panic.

* What are the next steps to see in the logs ? I cannot startup so what's the procedure to read the logs?
* I tried to restart the old kernel but that results in going to the mountroot prompt.
* Why does the server end in the mountroot prompt?
* What does he need to know from the mountroot?

Code:

Mounting from zfs:interno/ROOT/default failed with error 5

Loader variables:
  vfs.root.mountfrom=zfs:interno/ROOT/default

Manual root filesystem specification:
  <fstype>:<device> [option]
      Mount <device> using filesystem <fstype> and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/cd0 ro

?                      List valid disk bootdevices
.                       Yield 1 second (for background tasks)
<empty line>  Abort manual input

mountroot>

Best Regards,

R

tingo · Jan 14, 2020

Have you tried to read the errata? (Hint: it is where you found the release notes)

Relictrix · Jan 14, 2020

Hi Tingo,

Good idea, I did not. Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.

For your information, as I thought perhaps the upgrade failed. I rolled back to snapshot and gave it another shot. Of course after doing the upgrade again, it's exactely the same error. I now doubt that it was error 5 or 45. But at the end I come back to mountroot. I must say I don't really know what's the purpose of mountroot. I read some other threads here related to all kind of mountroot issues. But some people then give statements to execute, but not why. That's why i decided to make a new thread. My experience with mountroot is low , very low. I never used it before

. But the more trouble the more I learn. And I don't just want to try things on that machine, i still need that data.

SirDice · Jan 15, 2020

Relictrix said:
Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.

I'm going to bet it's this:

[2019-11-04] A late issue was discovered where systems using the graphics/drm-kmod port built on FreeBSD 12.0-RELEASE will crash during boot. It is advised to remove the port prior to upgrading, and build the port instead of using the upstream binary package.

Relictrix · Jan 15, 2020

Hi SirDice,

I will check that one soon , don't remember by heart if I installed it.

As your approach to finding this issue you really base yourself to the errata. I mean is there a way to find the error from the logs. I cannot open the 12.1 kernel, old kernel gives mountroot. Perhaps i could try to open the pool readonly and check some log. Or you got another, a better approach?

Best Regards,

R

SirDice · Jan 15, 2020

It's a common problem with users that have additional kernel modules installed. The packages in the repositories are still being built for 12.0. For 99% of the packages it's not a problem installing those on 12.1. Notable exceptions however are kernel modules, these will often cause the system to crash during boot.

This is one of the kernel modules that's been giving lots of users grief after they upgraded their systems. So it's my first guess it's been causing problems for you too.

Relictrix · Jan 15, 2020

Ok i see, I just noticed that i read the release notes not the errata. I want to go to 12.1, I will check the coming days and will respond back the result here. I already rolledback, will make a snapshot + backup externally. Then I will try again.

SirDice · Jan 15, 2020

Another common cause for crashes during boot after an upgrade (besides the aforementioned drm-kmod kernel module) are the VirtualBox kernel modules. So before you attempt the upgrade again make sure to disable any (third-party) kernel modules to avoid any potential issues. Kernel modules that come with the OS itself are not a problem.

Relictrix · Jan 16, 2020

Hi, the port graphics/drm-kmod was there, I deleted it, and run the upgrade. Will update here when the upgrade has finished.

Relictrix · Jan 16, 2020

Hi,

The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.

After restarting the pool in read only mode by starting the server in Single Boot mode via a USB disk, I could not see any files in /var/log or /var/crash. I wanted to search in detail for the error. Seems this got something to do with the mount, i mean normally there should be files there.

Just let me know if anybody has any thoughts, after a while I will probably rollback everything again because I need the server.

Best Regards,

R

SirDice · Jan 16, 2020

Relictrix said:
The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.

Do a pkg autoremove, graphics/drm-kmod is a so-called meta-port, it doesn't install anything of its own and pulls in one of the other drm-*-kmod ports/packages. And it's that dependency that actually has the kernel module.

Alternatively, just remove the kld_list from rc.conf. Or from loader.conf if you used that.

Relictrix · Jan 18, 2020

Hi,

I did that still does not work. Perhaps I have to be more drastic and should delete a lot more ports?

Best Regards,

R

tingo · Jan 19, 2020

First check that your zfs root pool is ok. Boot from a liveusb and check it.

Relictrix · Jan 19, 2020

My pool is ok if i rollback to snapshot. After the upgrade I cannot open my pool. I did one more time, this time with

Code:

pkg delete -a

followed by a

Code:

pkg autoremove

. Result is still an error.

More details about the error are: fatal trap 12: page fault while in kernel mode.

Relictrix · Jan 30, 2020

Hi, I can check my /etc/rc.conf next time I rollback again. But from what i remember, I don't have anything special in there. It's more or less a standard empty rc.conf.

Relictrix · Feb 23, 2020

This is my rc.conf

Code:

root@nuc1:~ # cat /etc/rc.conf
hostname="nuc1"
keymap="be.iso.kbd"
#ifconfig_em0="DHCP"
ifconfig_em0="DHCP"
#ifconfig_em0="inet 192.168.1.10 netmask 255.255.255.0"
#ifconfig_em0="inet 169.254.123.10 netmask 255.255.0.0"
defaultrouter=""
sshd_enable="YES"
ntpd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"

#firewall
ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"

#kde
#dbus_enable="YES"
#hald_enable="YES"
#sddm_enable="YES"

#xrdp_enable="YES"

#gnome
#dbus_enable="YES"
#hald_enable="YES"
#gdm_enable="YES"
#gnome_enable="YES"

#xfce
dbus_enable="YES"


#veracrypt

#virtualization
#vm_enable="YES"
#vm_dir="zfs:zroot/vms"
#vm_list="myubuntu"
#vm_delay="5"
#cloned_interfaces="bridge0 tap0"
#ifconfig_bridge0="addm em0 addm tap0"

#virtualization virtualbox
vboxnet_enable="YES"

#wireless
wlans_iwm0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"

bvansomeren · Feb 27, 2020

Hi,

Are you sure you're not using the Virtualbox drivers?

#virtualization virtualbox
vboxnet_enable="YES"

rootbert · Feb 27, 2020

I had a similar error, zfs_enable="YES" in rc.conf fixed it for me

Relictrix · Mar 6, 2020

Hi,

Just tried it. Commented the vboxnet_enable line and included the zfs_enable. Still the same kernel panic after upgrading from 12.0 to 12.1

Was a good attempt , I thought you found it.

Best Regards,

R

Relictrix · Mar 6, 2020

Hi,

Just did another attempt. This time started from a minimal /etc/rc.conf and from a minimal /boot/loader.conf.

The result is again a kernel panic.

Best Regards,

R

Relictrix · Mar 6, 2020

Hi,

Taking a picture the error is this:
Fatal trap 12: page fault while in kernel mode

Before the fatal trap i see the ipfilter is enabling, worth to disable this as well?
ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"

Best Regards,

rootbert · Mar 6, 2020

I would disable almost everything and gradually enable the services ... unfortunately costs time because of lots of reboots, but maybe you will find the problem.

Relictrix · Mar 8, 2020

Hi,

Finally! ipfilter was the root cause! After disabling that the upgrade worked.

Thanks rootbert for pointing out the zfs_enable="YES". I had to mount some points manually and while looking into detail setting on the mount itself: legacy and stuff, I saw the zfs_enable did the job.

This thread can be marked as solved.

freebsd_noob · Mar 8, 2020

Relictrix said:
This thread can be marked as solved.

You can do this on your own. Just edit your first post and add the prefix to the title bar.

Relictrix · Mar 8, 2020

Oh sorry, was not aware.

Thanks all for your reactions!

Solved Upgrade from 12.0 to 12.1 results in kernel panic

Member

Daemon

Member

Administrator

Member

Administrator

Member

Administrator

Member

Member

Administrator

Member

Daemon

Member

Member

Member

New Member

Well-Known Member

Member

Member

Member

Well-Known Member

Member

Active Member

Member