Hi,
I have one server acting as a host for VirtualBox-OSE. Not a long time ago I updated from 9.1 to 9.2 on 10.10.2013. The system worked fine for 10 days and on 19.10.2013 it crashed. I do not know the reason, there is nothing interesting in the log files. After a system update I rebuilt VB-OSE-kmod. I tried to use
Any ideas? Should I send the crash dump to the VirtualBox developers?
Dumps:
core.txt.1
I have one server acting as a host for VirtualBox-OSE. Not a long time ago I updated from 9.1 to 9.2 on 10.10.2013. The system worked fine for 10 days and on 19.10.2013 it crashed. I do not know the reason, there is nothing interesting in the log files. After a system update I rebuilt VB-OSE-kmod. I tried to use
kgdb to debug the dump but I was not able to find the reason.Any ideas? Should I send the crash dump to the VirtualBox developers?
Dumps:
uname -a
Code:
FreeBSD xbsdhost 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50:31 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 kldstat
Code:
Id Refs Address Size Name
1 29 0xffffffff80200000 15b93c0 kernel
2 1 0xffffffff817ba000 24708 geom_mirror.ko
3 2 0xffffffff817df000 50e30 pf.ko
4 1 0xffffffff81830000 12e58 if_bridge.ko
5 2 0xffffffff81843000 82c8 bridgestp.ko
7 3 0xffffffff81859000 17d08 netgraph.ko
8 3 0xffffffff81871000 5b298 vboxdrv.ko
9 1 0xffffffff81a12000 1583db zfs.ko
10 1 0xffffffff81b6b000 56f7 opensolaris.ko
11 1 0xffffffff81b71000 2bce pflog.ko
12 2 0xffffffff81b74000 2dad vboxnetflt.ko
13 1 0xffffffff81b77000 4446 ng_ether.ko
14 1 0xffffffff81b7c000 4046 vboxnetadp.ko gdb
Code:
Loaded symbols for /boot/modules/vboxnetadp.ko
#0 doadump (textdump=<value optimized out>) at pcpu.h:234
234 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) list *0xffffffff80923b6f
0xffffffff80923b6f is in callout_tickstofirst (/usr/src/sys/kern/kern_timeout.c:382).
warning: Source file is more recent than executable.
377 curticks = cc->cc_ticks;
378 while( skip < ncallout && skip < limit ) {
379 sc = &cc->cc_callwheel[ (curticks+skip) & callwheelmask ];
380 /* search scanning ticks */
381 TAILQ_FOREACH( c, sc, c_links.tqe ){
382 if (c->c_time - curticks <= ncallout)
383 goto out;
384 }
385 skip++;
386 }
(kgdb) backtrace
#0 doadump (textdump=<value optimized out>) at pcpu.h:234
#1 0xffffffff8090d486 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449
#2 0xffffffff8090d987 in panic (fmt=0x1 <Address 0x1 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:637
#3 0xffffffff80cf20d0 in trap_fatal (frame=0xc, eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:879
#4 0xffffffff80cf2431 in trap_pfault (frame=0xffffff80002ce870, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795
#5 0xffffffff80cf29e4 in trap (frame=0xffffff80002ce870) at /usr/src/sys/amd64/amd64/trap.c:463
#6 0xffffffff80cdbd13 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232
#7 0xffffffff80923b6f in callout_tickstofirst (limit=250) at /usr/src/sys/kern/kern_timeout.c:381
#8 0xffffffff80d68721 in getnextcpuevent (event=0xffffff80002ce9b0, idle=1) at /usr/src/sys/kern/kern_clocksource.c:282
#9 0xffffffff80d6a731 in cpu_idleclock () at /usr/src/sys/kern/kern_clocksource.c:791
#10 0xffffffff80ce2b61 in cpu_idle (busy=0) at /usr/src/sys/amd64/amd64/machdep.c:804
#11 0xffffffff809382ea in sched_idletd (dummy=<value optimized out>) at /usr/src/sys/kern/sched_ule.c:2625
#12 0xffffffff808db67f in fork_exit (callout=0xffffffff809381d0 <sched_idletd>, arg=0x0, frame=0xffffff80002ceb00) at /usr/src/sys/kern/kern_fork.c:992
#13 0xffffffff80cdc23e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606
#14 0x0000000000000000 in ?? ()
(kgdb) up
#1 0xffffffff8090d486 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449
warning: Source file is more recent than executable. last
Code:
# last
admin1991 pts/0 x.dyn.plu Sun Oct 20 00:22 still logged in
boot time Sun Oct 20 00:19
shutdown time Sun Oct 20 00:17
admin1991 pts/0 x.dyn.plu Sat Oct 19 23:57 - shutdown (00:20)
boot time Sat Oct 19 18:46 [highlight]<<<<----CRASH[/highlight]
admin1991 pts/0 x.dyn.plu Fri Oct 18 21:39 - 22:33 (00:54)
admin1991 pts/0 x.dyn.plu Thu Oct 17 20:31 - 20:39 (00:07)
admin1991 pts/0 x.dyn.plu Wed Oct 16 20:43 - 00:51 (04:08)
admin1991 pts/0 x.dyn.plu Tue Oct 15 21:49 - 21:51 (00:02)
admin1991 pts/0 x.dyn.plu Mon Oct 14 22:11 - 22:15 (00:04)
admin1991 pts/0 x.dyn.plu Sun Oct 13 20:54 - 21:00 (00:05)
admin1991 pts/0 x.dyn.plu Sat Oct 12 16:43 - 16:45 (00:01)
admin1991 pts/1 x.dyn.plu Sat Oct 12 16:26 - 16:43 (00:16)
admin1991 pts/0 x.dyn.plu Sat Oct 12 16:15 - 16:27 (00:11)
admin1991 pts/1 x.dyn.plu Thu Oct 10 20:45 - 21:34 (00:49)
admin1991 pts/0 x.dyn.plu Thu Oct 10 20:38 - 20:45 (00:07)
admin1991 pts/0 x.dyn.plu Thu Oct 10 16:32 - 16:33 (00:00)
admin1991 pts/1 x.dyn.plu Thu Oct 10 16:13 - 16:32 (00:19)
admin1991 pts/2 x.dyn.plu Thu Oct 10 16:03 - 16:15 (00:12)
admin1991 pts/0 x.dyn.plu Thu Oct 10 14:54 - 16:05 (01:11)
boot time Thu Oct 10 14:52
[highlight]shutdown time Thu Oct 10 14:51 <<<<9.1 --> 9.2[/highlight]
admin1991 pts/0 x.dyn.plu Thu Oct 10 14:09 - shutdown (00:41)
boot time Thu Oct 10 14:06
shutdown time Thu Oct 10 13:57
boot time Thu Oct 10 13:54
shutdown time Thu Oct 10 13:27
boot time Thu Oct 10 13:20
shutdown time Thu Oct 10 13:03
boot time Thu Oct 10 13:02
shutdown time Thu Oct 10 13:01
boot time Wed Oct 9 22:09
shutdown time Wed Oct 9 22:08
admin1991 pts/0 x.dyn.plu Wed Oct 9 21:39 - shutdown (00:28)
boot time Wed Oct 9 21:34
shutdown time Wed Oct 9 21:33
admin1991 pts/0 x.dyn.plu Wed Oct 9 19:59 - shutdown (01:34)
admin1991 pts/0 x Tue Oct 8 15:13 - 15:28 (00:14)
admin1991 pts/0 x.dyn.plu Mon Oct 7 23:15 - 23:23 (00:08)
admin1991 pts/0 x.dyn.plu Sun Oct 6 22:59 - 23:04 (00:04)
admin1991 pts/0 x.dyn.plu Fri Oct 4 21:07 - 21:48 (00:40)
admin1991 pts/0 x.dyn.plu Thu Oct 3 22:49 - 22:53 (00:03)
admin1991 pts/1 x Wed Oct 2 23:54 - 00:03 (00:08)
admin1991 pts/0 x Wed Oct 2 23:44 - 01:54 (02:10)
admin1991 pts/0 x Tue Oct 1 23:12 - 23:22 (00:09)core.txt.1
Code:
Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address<->= 0x4010
fault code<----><------>= supervisor read data, page not present
instruction pointer<--->= 0x20:0xffffffff80923b6f
stack pointer<-> = 0x28:0xffffff80002ce920
frame pointer<-> = 0x28:0xffffff80002ce960
code segment<--><------>= base 0x0, limit 0xfffff, type 0x1b
<------><------><------>= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags<------>= resume, IOPL = 0
current process><------>= 11 (idle: cpu3)
trap number<---><------>= 12
panic: page fault
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff80947986 at kdb_backtrace+0x66
#1 0xffffffff8090d9ae at panic+0x1ce
#2 0xffffffff80cf20d0 at trap_fatal+0x290
#3 0xffffffff80cf2431 at trap_pfault+0x211
#4 0xffffffff80cf29e4 at trap+0x344
#5 0xffffffff80cdbd13 at calltrap+0x8
#6 0xffffffff80d68721 at getnextcpuevent+0x141
#7 0xffffffff80d6a731 at cpu_idleclock+0x81
#8 0xffffffff80ce2b61 at cpu_idle+0x91
#9 0xffffffff809382ea at sched_idletd+0x11a
#10 0xffffffff808db67f at fork_exit+0x11f
#11 0xffffffff80cdc23e at fork_trampoline+0xe
Uptime: 9d3h50m43s
Dumping 1955 out of 32606 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_mirror.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/if_bridge.ko...Reading symbols from /boot/kernel/if_bridge.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_bridge.ko
Reading symbols from /boot/kernel/bridgestp.ko...Reading symbols from /boot/kernel/bridgestp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/bridgestp.ko
Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/modules/vboxdrv.ko...Reading symbols from /boot/modules/vboxdrv.ko.symbols...done.
done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/kernel/pflog.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/pflog.ko
Reading symbols from /boot/modules/vboxnetflt.ko...Reading symbols from /boot/modules/vboxnetflt.ko.symbols...done.
done.
Loaded symbols for /boot/modules/vboxnetflt.ko
Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ether.ko
Reading symbols from /boot/modules/vboxnetadp.ko...Reading symbols from /boot/modules/vboxnetadp.ko.symbols...done.
done.
Loaded symbols for /boot/modules/vboxnetadp.ko
#0 doadump (textdump=<value optimized out>) at pcpu.h:234
234<--->pcpu.h: No such file or directory.
<------>in pcpu.h
(kgdb) #0 doadump (textdump=<value optimized out>) at pcpu.h:234
#1 0xffffffff8090d486 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:449
#2 0xffffffff8090d987 in panic (fmt=0x1 <Address 0x1 out of bounds>)
at /usr/src/sys/kern/kern_shutdown.c:637
#3 0xffffffff80cf20d0 in trap_fatal (frame=0xc, eva=<value optimized out>)
at /usr/src/sys/amd64/amd64/trap.c:879
#4 0xffffffff80cf2431 in trap_pfault (frame=0xffffff80002ce870, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:795
#5 0xffffffff80cf29e4 in trap (frame=0xffffff80002ce870)
at /usr/src/sys/amd64/amd64/trap.c:463
#6 0xffffffff80cdbd13 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#7 0xffffffff80923b6f in callout_tickstofirst (limit=250)
at /usr/src/sys/kern/kern_timeout.c:381
#8 0xffffffff80d68721 in getnextcpuevent (event=0xffffff80002ce9b0, idle=1)
at /usr/src/sys/kern/kern_clocksource.c:282
#9 0xffffffff80d6a731 in cpu_idleclock ()
at /usr/src/sys/kern/kern_clocksource.c:791
#10 0xffffffff80ce2b61 in cpu_idle (busy=0)
at /usr/src/sys/amd64/amd64/machdep.c:804
#11 0xffffffff809382ea in sched_idletd (dummy=<value optimized out>)
at /usr/src/sys/kern/sched_ule.c:2625
#12 0xffffffff808db67f in fork_exit (
callout=0xffffffff809381d0 <sched_idletd>, arg=0x0,.
frame=0xffffff80002ceb00) at /usr/src/sys/kern/kern_fork.c:992
#13 0xffffffff80cdc23e in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:606
#14 0x0000000000000000 in ?? ()
(kgdb).