System won't boot after power outage - "can't load 'kernel'" (12.0 RELEASE amd64)

J

Julf

As the subject line says, my server didn't come up after a power outage, and refuses to boot, with the "can't load 'kernel'" message.
 
Things in general die when you suddenly turn them off and on again. Especially machines that have been running for quite some time 24/7. Are your disks still being detected? Is it possible your disk or your controller died due to the power spikes?
 
SirDice said:
Things in general die when you suddenly turn them off and on again. Especially machines that have been running for quite some time 24/7. Are your disks still being detected? Is it possible your disk or your controller died due to the power spikes?
Click to expand...

Looks like disks are being detected, and ls shows /boot exists
 
Ok, that's hopeful. As you mention ZFS, is this a full root-on-ZFS system? No encryption or anything like that?
 
It's a bit tricky as I don't exactly know what's wrong but try these on that loader prompt:
Code: 
set currdev=zfs:zroot/ROOT/default
boot /boot/kernel/kernel

The zroot/ROOT/default assumes you've used the default settings during installation and still have the 'default' boot environment.
 
SirDice said:
It's a bit tricky as I don't exactly know what's wrong but try these on that loader prompt:
Code: 
set currdev=zfs:zroot/ROOT/default
boot /boot/kernel/kernel

The zroot/ROOT/default assumes you've used the default settings during installation and still have the 'default' boot environment.
Click to expand...

Thanks!

Unfortunately that gives "Failed to load kernel '/boot/kernel/kernel' :(
 
Julf said:
ls shows /boot exists
Click to expand...
Does ls /boot/ or ls /boot/kernel/ show anything? Does ls /boot/ perhaps show a kernel.old? You could try that as a fallback. Anything to get it booting, we can fix any resulting kernel issues more easily later on. The loader(8) is a bit spartan.
 
SirDice said:
Does ls /boot/ or ls /boot/kernel/ show anything? Does ls /boot/ perhaps show a kernel.old? You could try that as a fallback. Anything to get it booting, we can fix any resulting kernel issues more easily later on. The loader(8) is a bit spartan.
Click to expand...

All 3 show up (/boot/, /boot/kernel/ and /boot/kernel.old/), with lots of files (mostly .ko), but boot /boot/kernel.old/kernel still gives can't load 'kernel'
 
I'm going to guess that this system was upgraded at some point in the past, and the zpool was upgraded, but the new bootloader was never installed. Running zpool upgrade shows a reminder message for a reason!

The only way I know of fixing this is to boot to external media (e.g. USB, network, etc.) that's running the same version of FreeBSD as your kernel/world, install the boot loader from that, and then reboot back to your normal system.
 
`Orum said:
I'm going to guess that this system was upgraded at some point in the past, and the zpool was upgraded, but the new bootloader was never installed. Running zpool upgrade shows a reminder message for a reason!
Click to expand...

You are probably right.

The only way I know of fixing this is to boot to external media (e.g. USB, network, etc.) that's running the same version of FreeBSD as your kernel/world, install the boot loader from that, and then reboot back to your normal system.
Click to expand...

Is there a way to install just the boot loader from the installer?
 
Is there a way to determine the version of the kernel on the hard disk from the installer?
 
Julf said:
If I boot using the boot image, the ZFS file system doesn't seem to be visible.
Click to expand...

This had me flummoxed - gpart show does show a freebsd-zfs partition, but zpool list results in no pools available. OK, zpool import shows it is there but in use by "other system", so clearly it was not shut down properly. Could that be why it doesn't boot?
 
Julf said:
so clearly it was not shut down properly.
Click to expand...
It has nothing to do with being cleanly shutdown or not. Normally ZFS pools need to be exported on one system before you can import them on another. In this case however you simply need to force the import.
 
SirDice said:
It has nothing to do with being cleanly shutdown or not. Normally ZFS pools need to be exported on one system before you can import them on another. In this case however you simply need to force the import.
Click to expand...

Thanks! Is forcing it safe?
 
SirDice said:
It's relatively safe to do, and there's no other way to get to the data to fix the booting issue.
Click to expand...

Importing it resulted in internal error: failed to initialize ZFS library, so I guess the imported zroot masks the zroot of the install image.

It also seems to have somehow corrupted the installer image, so I am now rewriting the USB stick (to try ZFS import under another name).
 
Julf said:
Importing it resulted in internal error: failed to initialize ZFS library, so I guess the imported zroot masks the zroot of the install image.

It also seems to have somehow corrupted the installer image, so I am now rewriting the USB stick (to try ZFS import under another name).
Click to expand...

Hmm... Even zpool import -f zroot jroot results in internal error: failed to initialize ZFS library for every ZFS command after the import - so it still replaces the install image root filesystem. There does seem to be a /boot/kernel directory, but no /boot/kernel/kernel.
 
Julf said:
I guess the zfs import still mounts the hard disk root over the installer root?
Click to expand...
Oh, yeah. That's definitely possible. Make use of the -R option to mount it on /tmp/temproot or some other place:
Code: 
         -R root
                 Sets the "cachefile" property to "none" and the "altroot"
                 property to "root"
 
SirDice said:
Oh, yeah. That's definitely possible. Make use of the -R option to mount it on /tmp/temproot or some other place:
Code: 
         -R root
                 Sets the "cachefile" property to "none" and the "altroot"
                 property to "root"
Click to expand...

Thanks! Is that an option to zpool import?
 
You must log in or register to reply here.
Back
Top