Why does my system hang?

samsam9988 · May 1, 2011

Hello,

With amd64, I don't know why my system hangs (for a while) when I started another program? The hanging is about 2 minutes or longer depending on how long the other program is running.

Here is another weird thing I found, the jdk16 sometimes occupied a cpu with > 100% usage.

Code:

last pid: 14432;  load averages:  0.75,  0.29,  0.15                                     up 1+07:15:02  20:45:09
87 processes:  2 running, 85 sleeping
CPU: 25.0% user,  0.0% nice,  0.0% system,  0.5% interrupt, 74.6% idle
Mem: 848M Active, 976M Inact, 758M Wired, 220K Cache, 776M Buf, 4817M Free
Swap: 4096M Total, 4096M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
13954 jboss      82  44    0  1010M   870M ucond   1   3:56 100.98% java
13309 jboss       1  44    0 38192K  5812K select  2   0:03  0.00% sshd
13310 jboss       1  44    0 20756K  3932K select  1   0:01  0.00% sftp-server
11535 jboss       1  76    0 10248K  2876K wait    3   0:00  0.00% bash
13919 jboss       1  76    0  8296K  1832K wait    0   0:00  0.00% sh
14428 jboss       1  44    0  9372K  2280K CPU1    1   0:00  0.00% top

Here is my hardware configuration:

Code:

Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.2-STABLE #0: Sat Apr 30 13:15:15 EST 2011
    [email]root@portal.ixsystems.com.au[/email]:/usr/obj/usr/src/sys/WEB64 amd64
module ath already present!
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Phenom(tm) II X4 955 Processor (3214.24-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f43  Family = 10  Model = 4  Stepping = 3
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  TSC: P-state invariant
real memory  = 8589934592 (8192 MB)
avail memory = 7706312704 (7349 MB)
ACPI APIC Table: <GBT    GBTUACPI>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
Security policy loaded: TrustedBSD MAC/portacl (mac_portacl)
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
..

I have recmpiled kernel with SMP enabled.

Can anyone please advise how to resolve the issue with this multi-tasking problem in my FreeBSD system?

Thanks
Sam

bestwc · May 3, 2011

Post your kernel configuration file here.

samsam9988 · May 4, 2011

bestwc said:
Post your kernel configuration file here.

Hello, thanks for the reply.
Here I attach my kernel config file.

Thanks
Sam

Beastie · May 4, 2011

You have that java process using most of the CPU time and almost a GB of memory and you're wondering why it ends up hanging the system?

The system is starved of resources. Kill the process ASAP.

samsam9988 · May 4, 2011

bestwc said:
Post your kernel configuration file here.

Even without jdk running, it hangs for few seconds when I login from another terminal.

samsam9988 · May 4, 2011

samsam9988 said:
Even without jdk running, it hang for few seconds when I login from another terminal.

And the jdk is not always running at >= 100%.

gkontos · May 4, 2011

Code:

last pid: 14432;  load averages:  0.75,  0.29,  0.15                                     up 1+07:15:02  20:45:09
87 processes:  2 running, 85 sleeping
[B]CPU: 25.0% user[/B],  0.0% nice,  0.0% system,  0.5% interrupt, 74.6% idle
Mem: 848M Active, 976M Inact, 758M Wired, 220K Cache, 776M Buf, 4817M Free
Swap: 4096M Total, 4096M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
13954 jboss      82  44    0  1010M   870M ucond   1   3:56 [B]100.98% java[/B]
13309 jboss       1  44    0 38192K  5812K select  2   0:03  0.00% sshd
13310 [B]jboss[/B]       1  44    0 20756K  3932K select  1   0:01  0.00% [B]sftp-server[/B]
11535 jboss       1  76    0 10248K  2876K wait    3   0:00  0.00% bash
13919 [B]jboss[/B]       1  76    0  8296K  1832K wait    0   0:00  0.00% [B]sh[/B]
14428 jboss       1  44    0  9372K  2280K CPU1    1   0:00  0.00% top

We can notice some interesting stuff here.

1) jboss only uses 1 core so there rest 3 are wasted. Memory looks good to me though.
2) jboss has a full working shell
3) jboss is running an sftp-server ?

That's a no go security wise.

What exactly do you mean that you system hangs when you start another program. Please explain what and how you are trying to start.
/var/log/messages is a good place also to look for problems.

samsam9988 · May 4, 2011

Here is the output of dmesg:

Code:

Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.2-STABLE #0: Sat Apr 30 13:15:15 EST 2011
    [email]root@portal.ixsystems.com.au[/email]:/usr/obj/usr/src/sys/WEB64 amd64
module ath already present!
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Phenom(tm) II X4 955 Processor (3214.24-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f43  Family = 10  Model = 4  Stepping = 3
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  TSC: P-state invariant
real memory  = 8589934592 (8192 MB)
avail memory = 7706312704 (7349 MB)
ACPI APIC Table: <GBT    GBTUACPI>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
Security policy loaded: TrustedBSD MAC/portacl (mac_portacl)
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <GBT GBTUACPI> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, afce0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xee00-0xeeff mem 0xd0000000-0xdfffffff,0xfdee0000-0xfdeeffff,0xfdd00000-0xfddfffff irq 18 at device
 5.0 on pci1
pci1: <multimedia, HDA> at device 5.1 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 5.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pci2: <serial bus, USB> at device 0.0 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 10.0 on pci0
pci3: <ACPI PCI bus> on pcib3
re0: <RealTek 8168/8111 B/C/CP/D/DP/E PCIe Gigabit Ethernet> port 0xce00-0xceff mem 0xfdfff000-0xfdffffff,0xfdff8000-0xfdffbfff irq 18 at
 device 0.0 on pci3
re0: Using 1 MSI-X message
re0: Chip rev. 0x28000000
re0: MAC rev. 0x00000000

miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus0
rgephy0:  10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX,
 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Ethernet address: 6c:f0:49:7f:fe:84
re0: [ITHREAD]
atapci0: <ATI IXP700/800 SATA300 controller> port 0xff00-0xff07,0xfe00-0xfe03,0xfd00-0xfd07,0xfc00-0xfc03,0xfb00-0xfb0f mem 0xfe02f000-
 0xfe02f3ff irq 22 at device 17.0 on pci0
atapci0: [ITHREAD]
atapci0: AHCI v1.10 controller with 6 3Gbps ports, PM supported
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ata4: <ATA channel 2> on atapci0
ata4: [ITHREAD]
ata5: <ATA channel 3> on atapci0
ata5: [ITHREAD]
ata6: <ATA channel 4> on atapci0
ata6: [ITHREAD]
ata7: <ATA channel 5> on atapci0
ata7: [ITHREAD]
ohci0: <OHCI (generic) USB controller> mem 0xfe02e000-0xfe02efff irq 16 at device 18.0 on pci0
ohci0: [ITHREAD]
usbus0: <OHCI (generic) USB controller> on ohci0
ohci1: <OHCI (generic) USB controller> mem 0xfe02d000-0xfe02dfff irq 16 at device 18.1 on pci0
ohci1: [ITHREAD]
usbus1: <OHCI (generic) USB controller> on ohci1
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfe02c000-0xfe02c0ff irq 17 at device 18.2 on pci0
ehci0: [ITHREAD]
usbus2: EHCI version 1.0
usbus2: <EHCI (generic) USB 2.0 controller> on ehci0
ohci2: <OHCI (generic) USB controller> mem 0xfe02b000-0xfe02bfff irq 18 at device 19.0 on pci0
ohci2: [ITHREAD]
usbus3: <OHCI (generic) USB controller> on ohci2
ohci3: <OHCI (generic) USB controller> mem 0xfe02a000-0xfe02afff irq 18 at device 19.1 on pci0
ohci3: [ITHREAD]
usbus4: <OHCI (generic) USB controller> on ohci3
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfe029000-0xfe0290ff irq 19 at device 19.2 on pci0
ehci1: [ITHREAD]
usbus5: EHCI version 1.0
usbus5: <EHCI (generic) USB 2.0 controller> on ehci1
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci1: <ATI IXP700/800 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfa00-0xfa0f at device 20.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
pci0: <multimedia, HDA> at device 20.2 (no driver attached)
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib4: <ACPI PCI-PCI bridge> at device 20.4 on pci0

fwohci0: <Texas Instruments TSB43AB23> mem 0xfdaff000-0xfdaff7ff,0xfdaf8000-0xfdafbfff irq 22 at device 14.0 on pci4
fwohci0: [ITHREAD]
fwohci0: OHCI version 1.10 (ROM=0)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:3c:70:b9:00:6c:f0:49
fwohci0: Phy 1394a available S400, 3 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0xafd0c000
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 02:3c:70:6c:f0:49
fwe0: Ethernet address: 02:3c:70:6c:f0:49
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 00:3c:70:b9:00:6c:f0:49 @ 0xfffe00000000, S400, maxrec 2048
fwohci0: Initiate bus reset
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode
ohci4: <OHCI (generic) USB controller> mem 0xfe028000-0xfe028fff irq 18 at device 20.5 on pci0
ohci4: [ITHREAD]
usbus6: <OHCI (generic) USB controller> on ohci4
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
atrtc0: <AT realtime clock> port 0x70-0x73 on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xcefff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
hwpstate0: <Cool`n'Quiet 2.0> on cpu0
Timecounters tick every 1.000 msec
firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0)  (me) 
firewire0: bus manager 0 
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 480Mbps High Speed USB v2.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 480Mbps High Speed USB v2.0
usbus6: 12Mbps Full Speed USB v1.0
ugen0.1: <ATI> at usbus0
uhub0: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <ATI> at usbus1
uhub1: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <ATI> at usbus2
uhub2: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ugen3.1: <ATI> at usbus3
uhub3: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <ATI> at usbus4
ugen6.1: <ATI> at usbus6
uhub6: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
acd0: DVDR <PIONEER DVD-RW DVR-218L/1.02> at ata2-master UDMA100 SATA 1.5Gb/s
ad8: 1907729MB <WDC WD20EARS-00J2GB0 80.00A80> at ata4-master UDMA100 SATA 3Gb/s
uhub6: 2 ports with 2 removable, self powered
uhub0: 3 ports with 3 removable, self powered
uhub1: 3 ports with 3 removable, self powered
uhub3: 3 ports with 3 removable, self powered
uhub4: 3 ports with 3 removable, self powered
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
GEOM: ad8s1: geometry does not match label (255h,63s != 16h,63s).
Root mount waiting for: usbus5 usbus2
Root mount waiting for: usbus5 usbus2
Root mount waiting for: usbus5 usbus2
uhub2: 6 ports with 6 removable, self powered
uhub5: 6 ports with 6 removable, self powered
Trying to mount root from ufs:/dev/ad8s1a
re0: link state changed to UP

Thanks
Sam

hainan · May 4, 2011

Did you add any options when installing java?

My openjdk6 install example:

First Install :

Code:

     DEBUG=off (default) "Enable legacy debugging support"
     FASTDEBUG=off (default) "Include fastdebug build"
     ICEDTEA=off (default) "Apply additional patches from IcedTea"
     IPV6=off (default) "Enable IPv6 support"
     [color="Red"]POLICY=on "Install the Unlimited Strength Policy Files"[/color]
     [color="Red"]SOUND=on "Enable sound support"[/color]
     TEST=off (default) "Add support for running regression test"
     [color="Red"]WEB=on "Enable the browser plugin and Java Web Start"
[/color]

with these selected options system hangs in 2 minutes when i test java web plugin. Then i deinstall openjdk6 and reinstall with

Code:

     DEBUG=off (default) "Enable legacy debugging support"
     FASTDEBUG=off (default) "Include fastdebug build"
     [color="Red"]ICEDTEA=on "Apply additional patches from IcedTea"[/color]
     IPV6=off (default) "Enable IPv6 support"
     [color="Red"]POLICY=on "Install the Unlimited Strength Policy Files"[/color]
     [color="Red"]SOUND=on "Enable sound support"[/color]
     TEST=off (default) "Add support for running regression test"
     [color="Red"]WEB=on "Enable the browser plugin and Java Web Start"[/color]

these options. After all system running stable.

samsam9988 · May 5, 2011

Hi, what are the differences between your old configuration and the new configuration?

Thanks
Sam

hainan · May 6, 2011

Hi

The differences between old and new config is only ICEDTEA option which is "Apply additional patches from IcedTea".

Code:

Old Config: ICEDTEA=off
New Config: ICEDTEA=[color="Red"]on[/color]

After ICEDTEA=on , browser plugin working very well.

I will try jdk16 later.

Best Regards

Hasan

Why does my system hang?

Attachments