Helping to improve vnet/vimage?

B

BobC

Hi Network Guru's-

I've been running 11-Release branch since 11 was well.... released. I think I'm on 11-RELEASE-p2. I compiled my own kernel to enable vimage and use iocage for jails. I am not using pf or doing anything "exotic" with networking (this is behind a separate firewall).

So far it's been mostly stable. But I have had 3 crashes, the 1st and third panic, I am pretty sure, were network related (savecore didn't work for panic #2 for some reason).

Both panic #1 and #3 messages were "Fatal trap 12: page fault while in kernel mode".

For panic #1 my best guess is something caused my mbufs to go crazy, netstat -m in my core.txt.0 file showed:

18446744073709549889/5011/3284/2035246 mbuf clusters in use (current/cache/total/max)

( "current" definitely looks suspicious ) But I could not determine *what* caused the mbuf madness.... assuming it was contributing factor.

For panic #3 the stack trace "seemed" to indicate problems with the netisr_dispatch(9) system call. But I could not determine more b/c nothing else looked out of the ordinary.

The stack trace is the same for both dumps.

Could this be related to Bug ID: 194515 ? But I'm not running pf...

Should I take this to the freebsd-net email alias?

Regardless, I am very interested in helping improve the vnet/vimage "feature".

What can I do to help improve vnet/vimage? Anything?

Bob
 
Ok... I'm going to pose the question before going to the email alias:

Is pf.ko required (even if I don't use BPF) when running with a vnet/vimage enabled kernel?

I have used kgdb on two of my crashes and they both end up at the exact same spot in the code.

Code: 
[root@tank /usr/obj/usr/src/sys]# kgdb MYKERNEL/kernel.debug /var/crash/vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
<6>vnet0:19: promiscuous mode enabled


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 06
fault virtual address    = 0x10
fault code        = supervisor read data, page not present
instruction pointer    = 0x20:0xffffffff80be6f6c
stack pointer            = 0x28:0xfffffe0838f87860
frame pointer            = 0x28:0xfffffe0838f878d0
code segment        = base 0x0, limit 0xfffff, type 0x1b
            = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags    = interrupt enabled, resume, IOPL = 0
current process        = 12 (swi1: netisr 0)
trap number        = 12
panic: page fault
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff80b25ac7 at kdb_backtrace+0x67
#1 0xffffffff80ada972 at vpanic+0x182
#2 0xffffffff80ada7e3 at panic+0x43
#3 0xffffffff80fb9d51 at trap_fatal+0x351
#4 0xffffffff80fb9f43 at trap_pfault+0x1e3
#5 0xffffffff80fb94ec at trap+0x26c
#6 0xffffffff80f9c861 at calltrap+0x8
#7 0xffffffff80bfe3bf at netisr_dispatch_src+0xff
#8 0xffffffff80be6754 at ether_input+0x54
#9 0xffffffff80bfecac at swi_net+0x1ac
#10 0xffffffff80a9465f at intr_event_execute_handlers+0x20f
#11 0xffffffff80a948c6 at ithread_loop+0xc6
#12 0xffffffff80a911c5 at fork_exit+0x85
#13 0xffffffff80f9cd9e at fork_trampoline+0xe
Uptime: 26d6h28m7s
Dumping 8142 out of 32648 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /usr/lib/debug//boot/kernel/coretemp.ko.debug...done.
done.
Loaded symbols for /boot/kernel/coretemp.ko
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/if_bridge.ko...Reading symbols from /usr/lib/debug//boot/kernel/if_bridge.ko.debug...done.
done.
Loaded symbols for /boot/kernel/if_bridge.ko
Reading symbols from /boot/kernel/bridgestp.ko...Reading symbols from /usr/lib/debug//boot/kernel/bridgestp.ko.debug...done.
done.
Loaded symbols for /boot/kernel/bridgestp.ko
Reading symbols from /boot/kernel/uplcom.ko...Reading symbols from /usr/lib/debug//boot/kernel/uplcom.ko.debug...done.
done.
Loaded symbols for /boot/kernel/uplcom.ko
Reading symbols from /boot/kernel/ucom.ko...Reading symbols from /usr/lib/debug//boot/kernel/ucom.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ucom.ko
Reading symbols from /boot/kernel/autofs.ko...Reading symbols from /usr/lib/debug//boot/kernel/autofs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/autofs.ko
Reading symbols from /boot/kernel/if_epair.ko...Reading symbols from /usr/lib/debug//boot/kernel/if_epair.ko.debug...done.
done.
Loaded symbols for /boot/kernel/if_epair.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
#0  doadump (textdump=<value optimized out>) at pcpu.h:221
221    pcpu.h: No such file or directory.
    in pcpu.h
(kgdb) list *0xffffffff80be6f6c
0xffffffff80be6f6c is in ether_nh_input (bpf.h:1487).
1482        struct bpf_if_ext *ext;
1483
1484        ext = (struct bpf_if_ext *)bpf;
1485        if (!LIST_EMPTY(&ext->bif_dlist))
1486            return (1);
1487        return (0);
1488    }
1489
1490    #define    BPF_TAP(_ifp,_pkt,_pktlen) do {                \
1491        if (bpf_peers_present((_ifp)->if_bpf))            \
Current language:  auto; currently minimal

Both crash dumps end in what appears to be a check for the existence of a BPF enable interface called from ether_nh_input() contained in /usr/src/sys/net/if_ethersubr.c line 517.

backtrace:

Code: 
kgdb) backtrace
#0  doadump (textdump=<value optimized out>) at pcpu.h:221
#1  0xffffffff80ada3f9 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366
#2  0xffffffff80ada9ab in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff80ada7e3 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:690
#4  0xffffffff80fb9d51 in trap_fatal (frame=0xfffffe0838f877b0, eva=16) at /usr/src/sys/amd64/amd64/trap.c:841
#5  0xffffffff80fb9f43 in trap_pfault (frame=0xfffffe0838f877b0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:691
#6  0xffffffff80fb94ec in trap (frame=0xfffffe0838f877b0) at /usr/src/sys/amd64/amd64/trap.c:442
#7  0xffffffff80f9c861 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236
#8  0xffffffff80be6f6c in ether_nh_input (m=<value optimized out>) at /usr/src/sys/net/if_ethersubr.c:517
#9  0xffffffff80bfe3bf in netisr_dispatch_src (proto=5, source=<value optimized out>, m=0xfffff804a1b71550) at /usr/src/sys/net/netisr.c:1121
#10 0xffffffff80be6754 in ether_input (ifp=<value optimized out>, m=0x0) at /usr/src/sys/net/if_ethersubr.c:759
#11 0xffffffff80bfecac in swi_net (arg=<value optimized out>) at /usr/src/sys/net/netisr.c:900
#12 0xffffffff80a9465f in intr_event_execute_handlers (p=<value optimized out>, ie=<value optimized out>) at /usr/src/sys/kern/kern_intr.c:1262
#13 0xffffffff80a948c6 in ithread_loop (arg=<value optimized out>) at /usr/src/sys/kern/kern_intr.c:1275
#14 0xffffffff80a911c5 in fork_exit (callout=0xffffffff80a94800 <ithread_loop>, arg=0xfffff80009372ac0, frame=0xfffffe0838f87ac0) at /usr/src/sys/kern/kern_fork.c:1038
#15 0xffffffff80f9cd9e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611
#16 0x0000000000000000 in ?? ()
(kgdb) quit

The problem is I can't seem to reproduce the condition to cause the panic...

Bob
 
BobC said:
Is pf.ko required (even if I don't use BPF) when running with a vnet/vimage enabled kernel?
Click to expand...
No, not that I know off. BPF on the other hand might be needed.

Judging by the issues you're having you may get more information posting to the mailing lists. There aren't a lot of developers on this board unfortunately (it's mostly for users, by users). I'm not sure which list would be best though, it's either freebsd-virtualization@ or freebsd-net@. I'd try the latter first.
 
You must log in or register to reply here.
Back
Top