Out of swap space error

Y

YuryG

Yesterday I've got "swap_pager: out of swap space" kernel message in the system log. Should I increase swap? It's an old machine with 8GiB of RAM and 4GiB of swap partition. (I assume the error occurred during devel/llvm-devel build.)
And, as far as I can see it, you cannot shrink a previous UFS partition, leaving only a late swap-file option, can you?
 
Is there a way to measure used RAM (and/ or swap) during some task like port build? I am not sure that machine without other tasks than building this port could ask for more than 12GiB (even if in 4 simultaneous jobs).
 
YuryG said:
Is there a way to measure used RAM (and/ or swap) during some task like port build? I am not sure that machine without other tasks than building this port could ask for more than 12GiB (even if in 4 simultaneous jobs).
Click to expand...
Building llvm in four threads will very likely exhaust 8+4 GB ram. If that llvm has fortran enabled, a single thread might need more than 8GB.
 
No Fortran nor debug is turned on. Four cores and four threads for the build.
I have increased swap. But I didn't notice previous builds of any port on that machine used at least half of it. So, it would be helpful to measure the real usage of RAM+swap without watching for top for quite a long time.
 
You can't really monitor physical memory usage per process since top(1) and friends do not display how much of a program's heap is in swapspace.

A good option is to run `vmstat 60` and watching the swapout column.

8 GB should build LLVM with 8 jobs, at least as it is contained in `make world`. Dunno whether the LLVM port takes more memory.

You will probably arrive at a lower total build time for the LLVM port by restricting parallel build jobs to your number of real cores and not using hyperthreading, while keeping swap usage low.
 
cracauer@ said:
You can't really monitor physical memory usage per process since top(1) and friends do not display how much of a program's heap is in swapspace.

A good option is to run `vmstat 60` and watching the swapout column.

8 GB should build LLVM with 8 jobs, at least as it is contained in `make world`. Dunno whether the LLVM port takes more memory.

You will probably arrive at a lower total build time for the LLVM port by restricting parallel build jobs to your number of real cores and not using hyperthreading, while keeping swap usage low.
Click to expand...
top has -w key if that is what I want. Is it? But real processes (clang ones) do not live long.


4 is the real number of cores and threads (no hyperthreading here). I can reduce used number with make -j 2, for example, and wait twice longer (approx.).
 
Oops, forgot that top has that now.

I am surprised that you ran out of 12 GB physical memory with a 4-parallel build of LLVM.
 
Here is data:

Code: 
$ for i in *.L/llvm*.stt; do echo "`cat $i | grep resident` `basename $i`"; done
   3720164  maximum resident set size llvm14.stt
  11361704  maximum resident set size llvm15.stt
   2322064  maximum resident set size llvm17.stt
   2695300  maximum resident set size llvm18.stt
   3618312  maximum resident set size llvm14.stt
  11181016  maximum resident set size llvm15.stt
   2213932  maximum resident set size llvm17.stt
   2603968  maximum resident set size llvm18.stt
$ for i in *llvm* ; do echo "$i: `cat $i/options | grep "^OPTIONS.*FLANG"`"; done
devel_llvm14: OPTIONS_FILE_SET+=FLANG
devel_llvm15: OPTIONS_FILE_SET+=FLANG
devel_llvm17: OPTIONS_FILE_UNSET+=FLANG
devel_llvm18: OPTIONS_FILE_UNSET+=FLANG

llvm18 reaches 2.6 GB (per thread), plus there is a kernel running, there is something stolen for the graphics, etc.etc. - and the devel version might be bigger.
(The data is from a simple "time -l" around my builds)
 
That is more than I expected. Can you check whether you have DEBUG on? Maybe the addition of Fortran defaults to a debug build?
 
cracauer@ said:
That is more than I expected. Can you check whether you have DEBUG on? Maybe the addition of Fortran defaults to a debug build?
Click to expand...
I don't have a DEBUG switch there. The fortran issue was discussed in some bug reports (and then switched off per default), it did break the builds quite often.
And I learned that paging is rather useless for multithreaded tasks, because they start to push each other out and make no progress.
 
You must log in or register to reply here.
Back
Top