Strange X crash related with the swap space

[piggy]

Hello, I don't know if this section of the forum is the right one for this post, then considering this crash happened on X and under X and seems related to swap space I decide to post here.

I have to say this box is absolutely, absolutely stable with very long up times and also in this case it was a program crashing and not the whole OS. Then, the strings in the log points to swap space and swap space is part of the system, so is it maybe something like a bug? I ask becouse this same system was rock solid doing pretty much the same stuff I was doing when the crash happened: KDE 4.5.3 environment, two terminals, one compiling code and the other scanning the system for updates, a web browser (chromium 6 with like 8 tabs opened). Nothing less, nothing more, like I did dozen times. The only variable here is in this case I used Chromium and usually I do use Opera (linux ports, becouse of the plugin support).

Here the OS details:

FreeBSD freebsd8vm 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 2010     root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

Here the crash details from the log:

Nov 19 20:31:13 freebsd8vm kernel: pid 10549 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:16 freebsd8vm kernel: pid 10548 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:16 freebsd8vm kernel: pid 10542 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:18 freebsd8vm kernel: pid 1361 (Xorg), uid 0, was killed: out of swap space
Nov 19 20:31:58 freebsd8vm kdm-bin[1138]: X server for display :0 terminated unexpectedly
Nov 19 20:32:08 freebsd8vm kernel: pid 10542 (npviewer.bin), uid 1002: exited on signal 11 (core dumped)
Nov 19 20:33:07 freebsd8vm kdm-bin[1138]: X server startup timeout, terminating
Nov 19 20:33:08 freebsd8vm kdm-bin[1138]: X server for display :0 cannot be started, session disabled
Nov 19 20:33:23 freebsd8vm kernel: pid 6016 (chrome), uid 1002: exited on signal 6 (core dumped)

Here the Top command results just a moment after the crash from another terminal:

last pid: 15405;  load averages:  0.25,  3.66,  3.99    up 1+15:28:06  20:36:26
53 processes:  1 running, 52 sleeping
CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
Mem: 97M Active, 23M Inact, 176M Wired, 6148K Cache, 110M Buf, 687M Free
Swap: 1943M Total, 20M Used, 1923M Free, 1% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
 1353 root          1  44    0  3820K   956K select   1:27  0.00% hald-addon-st
 1330 haldaemon     2  47    0 29812K  2916K piperd   0:56  0.00% hald
 1067 root          1  44    0  9384K  1200K select   0:20  0.00% nmbd
 1340 root          1  44    0  3820K   932K select   0:06  0.00% hald-addon-st
 1229 root          1  44    0  9996K  1136K select   0:04  0.00% sendmail
  766 root          1  44    0  8940K   812K select   0:02  0.00% syslogd
 1242 root          1  44    0  8968K   416K nanslp   0:01  0.00% cron
  919 messagebus    1  44    0  9096K  1356K select   0:01  0.00% dbus-daemon
  606 _dhcp         1  44    0  3288K   724K select   0:00  0.00% dhclient
  577 root          1  46    0  3288K   592K select   0:00  0.00% dhclient
 1333 root         17  44    0 30264K  2220K waitvt   0:00  0.00% console-kit-d
 1235 smmsp         1  44    0  9996K     0K pause    0:00  0.00% <sendmail>
15390 piggy           1  44    0 13524K  4368K select   0:00  0.00% sshd
15371 root          1  44    0 20592K  3228K select   0:00  0.00% hald-addon-mo
 1138 root          1  47    0  5076K  1332K select   0:00  0.00% kdm-bin

Now, as u can see, there was plenty of usefull swap space, so why even Xorg complained about "out of swap space" when there was a lot of swap space? Any idea?

 

[wblock@]

Here the crash details from the log:

Nov 19 20:31:13 freebsd8vm kernel: pid 10549 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:16 freebsd8vm kernel: pid 10548 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:16 freebsd8vm kernel: pid 10542 (npviewer.bin), uid 1002, was killed: out of swap space
Nov 19 20:31:18 freebsd8vm kernel: pid 1361 (Xorg), uid 0, was killed: out of swap space
...

Here the Top command results just a moment after the crash from another terminal:

last pid: 15405;  load averages:  0.25,  3.66,  3.99    up 1+15:28:06  20:36:26
53 processes:  1 running, 52 sleeping
CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
Mem: 97M Active, 23M Inact, 176M Wired, 6148K Cache, 110M Buf, 687M Free
Swap: 1943M Total, 20M Used, 1923M Free, 1% Inuse

Now, as u can see, there was plenty of usefull swap space, so why even Xorg complained about "out of swap space" when there was a lot of swap space? Any idea?

A moment after the crash was probably too late. The Flash player was killed and all the swap it had used was returned. By the time top updated, the swap showed free space again.

 

[piggy]

A moment after the crash was probably too late. The Flash player was killed and all the swap it had used was returned. By the time top updated, the swap showed free space again.

Mmmmm is the flash player such a hog and do Xorg is so prone to crash for a problem like that?

Here the snapshot of the system in this very moment (I reboot the machine also if the crash just killed the Xorg stuff). It is running pretty much the same tabs in Chrome as before and there are two terminals doing the same again:

last pid: 21712;  load averages:  0.97,  1.32,  1.16    up 0+00:28:38  21:34:46
197 processes: 2 running, 195 sleeping
CPU:  3.4% user,  0.0% nice,  4.9% system,  6.3% interrupt, 85.4% idle
Mem: 740M Active, 83M Inact, 119M Wired, 16M Cache, 110M Buf, 30M Free
Swap: 1943M Total, 44M Used, 1898M Free, 2% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
11589 max           1  46    0   256M 37088K select   0:53  0.98% npviewer.bin
16706 max           1  44    0   256M 37088K pcmwrv   0:04  0.98% npviewer.bin
 1335 root          1  44    0   120M 99852K select   1:00  0.00% Xorg
 1526 max          20  44    0   128M 47240K ucond    0:55  0.00% chrome
 1512 max           2  48    0   227M 37744K kqread   0:26  0.00% kdeinit4
 1559 max           2  44    0   112M 58636K kqread   0:22  0.00% chrome
 1573 max           1  45    0   266M   245M futex    0:18  0.00% npviewer.bin
 1555 max           2  44    0   132M 38280K kqread   0:17  0.00% chrome
 1454 max           3  56    0   302M 62752K select   0:15  0.00% kdeinit4
11595 max           1  44    0   256M 37088K futex    0:15  0.00% npviewer.bin
11567 max           1  44    0   266M   245M pcmwr    0:11  0.00% npviewer.bin
 1558 max           2  44    0   131M 33580K kqread   0:10  0.00% chrome
 1583 max           2  44    0   134M 44884K kqread   0:08  0.00% chrome
 1567 max           2  44    0   148M 58752K kqread   0:07  0.00% chrome
 1450 max           3  76    0   247M 42628K kqread   0:06  0.00% kwin
 1557 max           2  44    0   132M 42688K kqread   0:06  0.00% chrome

As u can see the swap space in use is just 2 per cent and the situation was identical as before. The only difference is before the system had like 5 or 6 days uptime and now it is freshly booted.

Are we sure is it not something related with bad handles in the way Freebsd kernel write on the swap file, even considering prolly the flash player is not that great code? I saw minor corruption of swap space in the past too, then I never had any type of application/subsystem crash for that like it happened this time.

 

[tingo]

The system is working as designed: if the system runs out of swap space (and more is requested) a process is killed in order to free some.
Make sure you have enough swap space for anything you will run on your system.

 

[piggy]

The system is working as designed: if the system runs out of swap space (and more is requested) a process is killed in order to free some.
Make sure you have enough swap space for anything you will run on your system.

This is first time something like this happen to me under Freebsd, and I started using it when Freebsd was at version 4. I have to reflect on the fact system treats X as any other program and it do not consider it is a critical program, becouse if it kills it for tight resources reason, it pretty could pretty much kill a lot of applications and possibly data running in the X session and this scenario can lead to a loose of sensible applications data :-(

Is it somewhere possible to tell kernel he should not kill X also if in need of resources?

 

[Beastie]

This is first time something like this happen to me under Freebsd, and I started using it when Freebsd was at version 4. I have to reflect on the fact system treats X as any other program and it do not consider it is a critical program

The "killing because out of swap space" behavior is FreeBSD acting normally. The "out of swap space" behavior is an application acting weird because of some serious bug.

this scenario can lead to a loose of sensible applications data

And not doing what it does could lead to a system failure and lockup with potential loss of filesystem consistency. Give me loss of "sensible applications data" anytime of the day.

Is it somewhere possible to tell kernel he should not kill X also if in need of resources?

I have no idea about any special configuration. But how could it not kill any application (X included) "if in need of resources"?
The situation is simple: you have a full memory, a full swap and 2 applications requesting more memory that isn't available. What should any sane system do? panic()? Or kill the offending application(s) and preserve overall stability?

 

[piggy]

The "killing because out of swap space" behavior is FreeBSD acting normally. The "out of swap space" behavior is an application acting weird because of some serious bug.

So u are saying, in my case, the common flash plugin application we all run on our systems (BSD and don't) is so badly written and can lead to a loss of data forcing operating system to kill resources becouse of it?

I have honestly to say I do use flash plugin most of the time in tons of Linux distros, on tight, old, even virtual machines and I never had such a problem. When the system become out of memory, it simply print an advise and refuse to start new applications. It doesn't kill a basic subsystem like X just becouse an app act weird and/or his swap file is going out of space.

IMHO, obviously.

This crash point me to think the way I do use and consider BSD usefull for, honestly it scared me. That day lucky I didn't had nothing really important opened on my desktop, often I do have very important documents, what happen if randomly OS shut them down just becouse of tight resources? I never saw such a behaviour, if this is a normal behaviour for a OS u would agree with me, computers will become useless becouse we can't trust them. Think if a machine dedicated to medical resources stops to monitor a patient just becouse is going out of space and so it kill some random apps vital for the patient! We can't simply handle this.

I repeat: this is the FIRST time it happened to me, I hope it never will, I hope it was some problems with my system (I doubt it), I hope all the problems I had in the past with Freebsd swap was problems related with my system (I doubt it), then surely I can't see this behaviour as a solid OS design, and yes it scare me. Unless there is a method I can control it.

Simply I can't see the point for an OS to stay up if it kill the reason becouse he exist: running applications.