Cannot mount zroot after recompiling kernel

dougs

dougs

I saw that there was a security advisory for 9.1-RELEASE and wanted to upgrade to P6. My system has two ZFS partitions - one a single disk zroot and the other a three disk array called zdata. I used the following link to create these two ZFS partitions and have been using the system in production for roughly a year by now:

https://www.dan.me.uk/blog/2012/01/22/booting-from-zfs-raid0156-in-freebsd-9-0-release/

However, after following these steps:

Code: 
cd /usr/src
make buildworld
make kernel KERNCONF=FreeBSDzfs
shutdown -r now
<boot in single user mode>
adjkerntz -i
mount -u /
zfs mount -a
mergemaster -Ui -p
cd /usr/src
make installworld
mergemaster -Ui
make delete-old
make delete-old-libs
reboot

I found myself staring at this error:

Code: 
Trying to mount root from zfs:root []...
Aug 26 09:09:14 init: NSSWITCH(_nsdispatch): winbind, passwd, endpwent, not found
and no fallback provided
Enter full pathname and shell or RETURN for /bin/sh:
Cannot read termcap database;
using dumb terminal settings.
#

zfs get mountpoint gives me the configuration information for both zroot and zdata so that looks good. /boot/loader.conf has the necessary entries for mounting zroot.

Do I need to export and import ZFS configuration as follows:

Code: 
cd /boot/zfs
zpool export zroot && zpool import zroot
cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache
zfs unmount -a
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var

This is a new experience and I want to make sure I do not destroy the zroot partition nor do I want to destroy any important boot-related files/configuration. Could someone assist me in recreating the correct boot environment? And more importantly, how to avoid doing this again?

~Doug
 
The error is not related to ZFS. I'm guessing you forgot to include something in your kernel configuration. Can you post it so we can have a look?
 
Content of /usr/src/sys/i386/conf/FreeBSDzfs kernel configuration file:
Code: 
#
# GENERIC -- Generic kernel configuration file for FreeBSD/i386
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#    http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD: releng/9.0/sys/i386/conf/GENERIC 227305 2011-11-07 13:40:54Z marius $

include         GENERIC
nocpu           I486_CPU
nocpu           I586_CPU
ident           FreeBSDzfs

options         KVA_PAGES=512   # increase system memory usage from 1GB to 2GB for ZFS usage

nodevice        eisa
nodevice        fdc

Content of /boot/loader.conf:
Code: 
zfs_load="YES"
vfs.root.mountfrom="zfs.zroot"
vfs.zfs.prefetch_disable="1"

vm.kmem_size="1536M"
vm.kmem_size_max="2048M"
vfs.zfs.arc_max="768M"
vfs.zfs.arc_min="128M"
vfs.zfs.vdev.cache.size="20M"
vfs.zfs.write_limit_override=268435456

kern.maxvnodes=250000
aio_load="YES"
autoboot_delay="5"
hw.em.rxd=4096
hw.em.txd=4096

Anything appear amiss?

~Doug
 
Code: 
# zfs get mountpoint
NAME             PROPERTY     VALUE        SOURCE
zdata            mountpoint   /zdata       default
zdata/home       mountpoint   /zdata/home  default
zroot            mountpoint   /            local
zroot/swap       mountpoint   -            -
zroot/usr        mountpoint   /usr         inherited from zroot
zroot/usr/ports  mountpoint   /usr/ports   inherited from zroot
zroot/usr/src    mountpoint   /usr/src     inherited from zroot
zroot/var        mountpoint   /var         inherited from zroot
zroot/var/log    mountpoint   /var/log     inherited from zroot
 
I suddenly remembered that I left out one step while compiling the kernel. I typed nextboot -o "-s" -k kernel right after installing the kernel. Would this cause any issue?

~Doug
 
Egads.

After fruitlessly rebooting and getting stuck in single-user mode, and poking around in the boot directories trying to verify that everything is as they should be, I typed exit. Lo and behold, the system started loading normally and booted normally bringing up the command prompt at the end!

I rebooted two more times to make sure that it would boot up normally and each time it did.

How the heck did I end up in single user mode the first few times before I broke out?

More confused than ever!

~Doug
 
nextboot -o "-s" -k kernel does not work as expected on ZFS on root installations. The /boot/nextboot.conf is written as expected and the system will boot into single user mode without having to select single user mode from the boot menu. However, the /boot/nextboot.con won't be removed automatically, you have to delete it yourself or your system will boot into single user mode again on next boot.
 
/boot/nextboot.conf wasn't in /boot thus was able to reboot normally each time. I'm going to hazard a guess that when I typed exit from within single-user mode, that file got removed from /boot.

I also saw this in my terminal console screen which I left hanging upon rebooting:
Code: 
[14]root@test:/usr/src # nextboot -o "-s" -k kernel
WARNING: loader(8) has only R/O support for ZFS
nextboot.conf will NOT be reset in case of kernel boot failure
[15]root@test:/usr/src # exit
logout

Looks like I shouldn't use nextboot as part of my kernel compilation routine on systems using ZFS in root partitions.

I will chalk this up as a learning experience!

~Doug
 
You must log in or register to reply here.
Back
Top