Currently 6G is swapped out on my desktop PC , but there is something newer. zswap.

Alain De Vos said:
I just need to compile two ports, vscode&chromium in poudriere with TMPFS all, and my system uses more than 16G of memory ...
Click to expand...
You can simply reduce the number of parallel jobs in MAKEOPTS and/or EMERGE_DEFAULT_OPTS to stay under 16GB RAM usage during compilation.

What interests me: why don't you install it via pkg? Chromium binary packages are available on almost all systems, including macOS and Linux, and they work fine.
 
According to Mr. Spock it's i highly illogical i do things difficult when i can them do simple.
PS: Currently compiling only Chromium&Iridium browser, ie two ports with poudriere & TMPFS setting "yes".
16G memory & 19G swap usage. Foreseen end time 5 hours.
[My CPU is Intel 12-th generation ]
On my older PC compiling Chromium took 35 hours. With these compile times if you can do something parallel you do it.
 
Alain De Vos said:
According to Mr. Spock it's i highly illogical i do things difficult when i can them do simple.
PS: Currently compiling only Chromium&Iridium browser, ie two ports with poudriere & TMPFS setting "yes".
16G memory & 19G swap usage. Foreseen end time 5 hours.
[My CPU is Intel 12-th generation ]
On my older PC compiling Chromium took 35 hours. With these compile times if you can do something parallel you do it.
Click to expand...
Are you compiling both apps at the same time? I think that is quite a long time for your hardware.
I used to use Arch Linux for a long time and let's just say I had +- 12 AUR packages that had to compile regularly on very weak hardware.
I've never seen anything take longer than 45 minutes.
It is apparently true that that large C++ programs have extremely slow compilation times for quite a while.
I do wonder why C++ is so popular if it causes such compilation problems.
Again I suspect you can get faster performance by making sure no SWAP is used by cutting down the parallel processes.

Knowledge Base:Emerge out of memory - Gentoo wiki

wiki.gentoo.org wiki.gentoo.org
  • The system became extremely slow because of swap usage while emerge something.
  • It is not advised to use parallel jobs in either MAKEOPTS nor EMERGE_DEFAULT_OPTS on systems that do not have much RAM (Raspberry Pi's with 512 MB of RAM, old desktop computers, etc.).
  • Try to lower number of parallel jobs for some packages which usually requires more RAM to compile.
 
You are right. Too much swapping influences performance bad.
In poudriere i configured,
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX
# data - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all - Run the entire build in memory, including builder jails.
# yes - Enables tmpfs(5) for wrkdir and data
# no - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS="yes"

PARALLEL_JOBS=13
PREPARE_PARALLEL_JOBS=17
In make.conf,
MAKE_JOBS_NUMBER=12

Number of cores on my CPU is 12.
A good measurement of frequency of pages swapped-in would be interesting. Because pages swapped in is expensive on load.

I'll try now with,
MAKE_JOBS_NUMBER=6
This number seem to have hardly an influence on swap usage ...

Trying now,
USE_TMPFS="wrkdir"
Not better swap usage, 16G

====================================================================================
Trying something new,
PARALLEL_JOBS=1
PREPARE_PARALLEL_JOBS=11
Meaning i build packages not in parallel but one after another. When done in memory it gives good performance...
With these settings swap usage dropped from 19G to 8G.
 
How many threads does your CPU have?
You still use a lot of SWAP. I would give this option a try:

MAKEOPTS="-j8"
NINJAOPTS=-j8

Or if this still uses a lot of swap I would try this for example:
MAKEOPTS="-j4"
NINJAOPTS=-j4

You can export those values with the export command.
Another thing often mentioned to reduce RAM usage is 'disabling jumbo-build'.
 
(Gentoo)MAKEOPTS=(FreeBSD)MAKE_JOBS_NUMBER

##
# MAKE_JOBS_SAFE
# - This port can safely be built on multiple cpus in parallel.
# The make will be invoked with -jX parameter where X equals
# number of cores present in the system.
# MAKE_JOBS_UNSAFE
# - Disallow multiple jobs even when user set a global override.
# To be used with known bad ports.
# DISABLE_MAKE_JOBS
# - Set to disable the multiple jobs feature. User settable.
# FORCE_MAKE_JOBS
# - Force all ports to be built with multiple jobs, except ports
# that are explicitly marked MAKE_JOBS_UNSAFE. User settable.
# MAKE_JOBS_NUMBER
# - Override the number of make jobs to be used. User settable.


You may need to do the combination of the following two values:
FORCE_MAKE_JOBS=yes
MAKE_JOBS_NUMBER=8

It seems you are not using the bold option.
What I wonder is what the FreeBSD equivalent is for NINJAOPTS?
 
For whom is interested, my make.conf:
Code: 
MYFLAGS="-fno-lto -O2 -pipe"
CFLAGS+="${MYFLAGS}"
CXXFLAGS+="${MYFLAGS}"
BATCH=yes
CPUTYPE?=alderlake
DISABLE_LICENSES=yes
MAKE_JOBS_NUMBER=8
MAKE_JOBS_UNSAFE=yes
MTREE_FOLLOWS_SYMLINKS= -L
NO_CHECKSUM=yes
OPENSSLBASE=/usr/local
WITHOUT_CCACHE_BUILD=yes
WITHOUT_MANCOMPRESS=yes

DEFAULT_VERSIONS+= mysql=10.6m
DEFAULT_VERSIONS+= ssl=openssl

MAKEOPTS="-j8"
NINJAOPTS="-j8"

& poudriere.conf
Code: 
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS="wrkdir data"

PARALLEL_JOBS=6
PREPARE_PARALLEL_JOBS=11
# By default MAKE_JOBS is disabled to allow only one process per cpu
# Use the following to allow it anyway
ALLOW_MAKE_JOBS=yes
 
You must log in or register to reply here.
Back
Top