Solved Kernel panic when starting jail via iocage with VIMAGE/VNET

Hello,

the problem I describe below has to do with a FreeBSD Virtual Machine running 10.1-RELEASE-p29 (build from source). The hypervisor is KVM (Proxmox VE 3.4).

This is a test of looking if I could get the following to work:
  • have a ZFS ZPOOL
  • for running jails with VIMAGE/VNET functionality
  • using the iocage package to provision and manage the jails
I've build a custom kernel with the following configuration:
Code:
include GENERIC

cpu             HAMMER
ident           MYKERNEL

# Firewalling
#device          pf
#device          pflog

# Virtual networking for jail
nooptions       SCTP        # 8.0-RELEASE does not support SCTP with VIMAGE
options         VIMAGE
options         RACCT
options         RCTL
device          epair
device          if_bridge

# The nullFS to mount local directory
options         NULLFS

The rc.conf of this VM is:
Code:
ifconfig_em0="inet 172.16.1.231 netmask 255.255.255.0"
hostname="zfstest.intra.socruel.nu"
defaultrouter="172.16.1.1"
kern_securelevel_enable="NO"
kern_securelevel="0"
log_in_vain="1"
icmp_drop_redirect="YES"
icmp_log_redirect="YES"
sshd_enable="YES"
ntpdate_enable="YES"
ntpdate_hosts="194.109.20.18"
accounting_enable="YES"
salt_minion_enable="YES"
ntpd_enable="NO"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
portmap_enable="NO"
inetd_enable="NO"
update_motd="NO"
dumpdev="AUTO"
zfs_enable="YES"
cloned_interfaces="bridge0 bridge1"
ifconfig_bridge0="addm em1 up"
ifconfig_bridge1="addm em2 up"
ifconfig_em1="up"
ifconfig_em2="up"

The ZPOOL, called sctest, is healthy:
lrsw@zfstest conf $ sudo zpool status sctest

Code:
  pool: sctest
state: ONLINE
  scan: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        sctest        ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            gpt/zfs1  ONLINE       0     0     0
            gpt/zfs2  ONLINE       0     0     0
          mirror-1    ONLINE       0     0     0
            gpt/zfs3  ONLINE       0     0     0
            gpt/zfs4  ONLINE       0     0     0

errors: No known data errors

The commands used to create the test jail are:
Code:
% sudo iocage create tag=jailtest
% sudo iocage set defaultrouter=172.16.1.1 jailtest
% sudo iocage set ipv4_addr="vnet0|172.16.1.232/24" jailtest

So far so good. But when I start this jail with the command sudo iocage start jailtest the VM crashes because of a kernel panic. The information from the /var/log/messages:
Code:
Feb 19 09:53:00 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/sbin/zpool status sctest
Feb 19 09:55:07 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/usr/local/sbin/iocage destroy
ALL
Feb 19 09:56:43 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/usr/local/sbin/iocage create
tag=jailtest pkglist=/usr/local/etc/iocage/pkgs.txt
Feb 19 09:59:32 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/usr/local/sbin/iocage set def
aultrouter=172.16.1.1 jailtest
Feb 19 09:59:45 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/usr/local/sbin/iocage set ip4
_addr=vnet0|172.16.1.232/24 jailtest
Feb 19 10:00:00 zfstest sudo:     lrsw : TTY=pts/0 ; PWD=/home/lrsw ; USER=root ; COMMAND=/usr/local/sbin/iocage start jailtest
Feb 19 10:00:02 zfstest devd: Executing '/etc/pccard_ether epair0a start'

Feb 19 10:00:02 zfstest kernel:
Feb 19 10:00:02 zfstest kernel:
Feb 19 10:00:02 zfstest kernel: Fatal trap 12: page fault while in kernel mode
Feb 19 10:00:02 zfstest kernel: cpuid = 0; apic id = 00
Feb 19 10:00:02 zfstest kernel: fault virtual address   = 0x0
Feb 19 10:00:02 zfstest kernel: fault code              = supervisor read instruction, page not present
Feb 19 10:00:02 zfstest kernel: instruction pointer     = 0x20:0x0
Feb 19 10:00:02 zfstest kernel: stack pointer           = 0x28:0xfffffe0120912900
Feb 19 10:00:02 zfstest kernel: frame pointer           = 0x28:0xfffffe0120912950
Feb 19 10:00:02 zfstest kernel: code segment            = base 0x0, limit 0xfffff, type 0x1b
Feb 19 10:00:02 zfstest kernel: = DPL 0, pres 1, long 1, def32 0, gran 1
Feb 19 10:00:02 zfstest kernel: processor eflags        = interrupt enabled, resume, IOPL = 0
Feb 19 10:00:02 zfstest kernel: current process         = 4655 (ifconfig)
Feb 19 10:01:16 zfstest syslogd: kernel boot file is /boot/kernel/kernel
Feb 19 10:01:16 zfstest kernel: trap number             = 12
Feb 19 10:01:16 zfstest kernel: panic: page fault
Feb 19 10:01:16 zfstest kernel: cpuid = 0
Feb 19 10:01:16 zfstest kernel: KDB: stack backtrace:
Feb 19 10:01:16 zfstest kernel: #0 0xffffffff80967ee0 at kdb_backtrace+0x60
Feb 19 10:01:16 zfstest kernel: #1 0xffffffff8092caa5 at panic+0x155
Feb 19 10:01:16 zfstest kernel: #2 0xffffffff80cd9ecf at trap_fatal+0x38f
Feb 19 10:01:16 zfstest kernel: #3 0xffffffff80cda1e8 at trap_pfault+0x308
Feb 19 10:01:16 zfstest kernel: #4 0xffffffff80cd984a at trap+0x47a
Feb 19 10:01:16 zfstest kernel: #5 0xffffffff80cbf722 at calltrap+0x8
Feb 19 10:01:16 zfstest kernel: #6 0xffffffff809f0fb3 at ifioctl+0x10e3
Feb 19 10:01:16 zfstest kernel: #7 0xffffffff8097fa15 at kern_ioctl+0x255
Feb 19 10:01:16 zfstest kernel: #8 0xffffffff8097f76c at sys_ioctl+0x13c
Feb 19 10:01:16 zfstest kernel: #9 0xffffffff80cda801 at amd64_syscall+0x351
Feb 19 10:01:16 zfstest kernel: #10 0xffffffff80cbfa0b at Xfast_syscall+0xfb
Feb 19 10:01:16 zfstest kernel: Uptime: 29m22s
Feb 19 10:01:16 zfstest kernel: Dumping 605 out of 4072 MB:..3%..11%..22%..32%..43%..51%..61%..72%..82%..93%
Feb 19 10:01:16 zfstest kernel: Dump complete

Feb 19 10:01:16 zfstest kernel: Automatic reboot in 15 seconds - press a key on the console to abort
Feb 19 10:01:16 zfstest kernel: Rebooting...
Feb 19 10:01:16 zfstest kernel: Copyright (c) 1992-2014 The FreeBSD Project.
Feb 19 10:01:16 zfstest kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Feb 19 10:01:16 zfstest kernel: The Regents of the University of California. All rights reserved.
Feb 19 10:01:16 zfstest kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Feb 19 10:01:16 zfstest kernel: FreeBSD 10.1-RELEASE-p29 #0 r295763: Fri Feb 19 09:16:53 CET 2016
Feb 19 10:01:16 zfstest kernel: root@zfstest.intra.socruel.nu:/usr/obj/usr/src/sys/MYKERNEL amd64

Can anyone tell from the above what causes this? Did I something wrong? I know that VIMAGE/VNET is still experimental in 10.x but users are using it. It is on the reoadmap for 11 to make it stable and fix all the issues. But I would like to test this now.

Thank you in advance for any feedback.

Regards,
Lars.
 
It's unlikely this issue will be fixed in 10.1 as it will be End-of-Life by the end of the year. And apparently the issue has been resolved on 10.3.
 
Back
Top