System boot hangs on trying to mount a zfs pool

Hi

I have a problem with one of the servers - system boot hangs on trying to mount a ZFS pool:

Code:
Trying to mount root from zfs:tank/root

Fatal double fault:
eip = 0xc0ad0d19
esp = 0xc5b93ff8
ebp = 0xc5b94080
cpuid = 0; apic id = 00
panic: duble fault
cpuid = 0
KDB: stack backtrace:
#0 0xc0ae102b at kdb_backtrace_0x4b
#1 0xc0aad2d4 at panic+0x1a4
#2 0xc0e4da90 at cpu_fetch_syscall_args+0
Uptime 6s
Automatic reboot in 15 seconds - press a key on the console to abort

The problem is temporarily removed after the boot server from FreeBSD-9.1-RELEASE-i386-memstick.img and importing the pool.

Server specifications:

Code:
CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (3000.25-MHz 686-class CPU)
...
real memory  = 2147483648 (2048 MB)
avail memory = 2073387008 (1977 MB)
...
ada0 at ata3 bus 0 scbus1 target 0 lun 0
ada0: <WDC WD2002FAEX-007BA0 05.01D05> ATA-8 SATA 3.x device
ada0: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad6
ada1 at ata4 bus 0 scbus2 target 0 lun 0
ada1: <ST32000641AS CC13> ATA-8 SATA 3.x device
ada1: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad8

Code:
# uname -a
FreeBSD local.domain.net 9.1-RELEASE-p6 FreeBSD 9.1-RELEASE-p6 #8: Fri Aug 23 09:58:42 EEST 2013     root@local.domain.net:/usr/obj/usr/src/sys/MYKERNEL  i386

# cat /usr/src/sys/i386/conf/MYKERNEL
# add Generic kernel
include    GENERIC

ident      MYKERNEL

# IPFW with forwarding and divert
options    IPFIREWALL
options    IPFIREWALL_VERBOSE
options    IPFIREWALL_VERBOSE_LIMIT=5
options    IPFIREWALL_DEFAULT_TO_ACCEPT
options    IPDIVERT
options    IPFIREWALL_FORWARD
options    IPFIREWALL_NAT
options    LIBALIAS

# Dummynet
options    DUMMYNET
options    HZ=1000

# QUAGGA with MD5
options TCP_SIGNATURE
options IPSEC
device crypto

Code:
# zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
tank  1.81T   550G  1.28T    29%  1.00x  ONLINE  -

# zpool status
  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 1h35m with 0 errors on Mon May 20 13:59:02 2013
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    ada1p3  ONLINE       0     0     0
	    ada0p3  ONLINE       0     0     0

errors: No known data errors

# zfs list
NAME                            USED  AVAIL  REFER  MOUNTPOINT
tank                            550G  1.25T    22K  none
tank/root                       550G  1.25T   449M  /
tank/root/backup               18.1M  1.25T  18.1M  /backup
tank/root/smb                   543G  1.25T   543G  /smb
tank/root/tmp                  5.19M  1.25T  5.19M  /tmp
tank/root/usr                  6.75G  1.25T  4.24G  /usr
tank/root/usr/home             17.0M  1.25T  17.0M  /usr/home
tank/root/usr/ports            1.09G  1.25T   301M  /usr/ports
tank/root/usr/ports/distfiles   818M  1.25T   818M  /usr/ports/distfiles
tank/root/usr/ports/packages     31K  1.25T    31K  /usr/ports/packages
tank/root/usr/src              1.40G  1.25T  1.40G  /usr/src
tank/root/var                   176M  1.25T  10.1M  /var
tank/root/var/crash            31.5K  1.25T  31.5K  /var/crash
tank/root/var/db                161M  1.25T   155M  /var/db
tank/root/var/db/pkg           5.91M  1.25T  5.91M  /var/db/pkg
tank/root/var/empty              22K  1.25T    22K  /var/empty
tank/root/var/log              4.12M  1.25T  4.12M  /var/log
tank/root/var/mail             1.17M  1.25T  1.17M  /var/mail
tank/root/var/run                61K  1.25T    61K  /var/run
tank/root/var/tmp              82.5K  1.25T  82.5K  /var/tmp


Code:
# cat /boot/loader.conf
# USB
hw.usb.no_pf=1

# GEOM
kern.geom.raid.enable="0"

# ZFS
zfs_load="YES"
vfs.root.mountfrom="zfs:tank/root"
vm.kmem_size="512M"
vm.kmem_size_max="512M"

The system was installed with the image of 9.0-RELEASE-i386 special edition. Now the system is upgraded to 9.1-RELEASE-p6.

How to permanently fix the problem?
 
Do you have a possibility to migrate this server to a machine that can use the AMD64 version of FreeBSD? Support for ZFS on i386 isn't as good as one would like.
 
Unfortunately this is not possible yet. I suspect that the case in architecture, but there was hope for the possibility of more or less stable operation at i386.
 
I got the same problem quite often. I do not know the reason. Recently, I changed a keyboard. Now it is working. Is it possible that we changed the device after the kernel is recompiled? I will confirm it to compile the kernel with ps2 keyboard then use a USB keyboard to boot to trigger the crash.
 
Back
Top