Poudriere spawns only one make process per building-port.

# By default MAKE_JOBS is disabled to allow only one process per cpu
# Use the following to allow it anyway
ALLOW_MAKE_JOBS=YES

Does not seem to work.
 
Sure it does. Most likely, you're trying to build a port that has MAKE_JOBS_UNSAFE set. This is only set for good reasons (the upstream build system is broken and can't correctly handle parallel make). When it's set, this port will never use parallel make.

A bit less likely, but also possible: Upstream uses a build system that just doesn't support parallel make.
 
So if I try:

Poudriere.conf:
PARALLEL_JOBS=1000
PREPARE_PARALLEL_JOBS=1000
ALLOW_MAKE_JOBS=YES


make.conf:
MAKE_JOBS_NUMBER=1000

Then 1000 make jobs are spawn when I build, let's say , LLVM11 , with poudriere ?
----------------------
I just performed the test, only 1 make job is running.
ps aux | grep make
root 45692 0.0 0.0 12960 3536 0 IJ 08:45 0:00.06 /usr/bin/make -C /usr/ports/devel/llvm11 build
root 25825 0.0 0.0 12948 2052 6 S+ 08:47 0:00.00 grep make
root@aladin:/home/x #
 
poudriere doesn't use the system's make.conf but constructs one from the files you place in /usr/local/etc/poudriere.d. Then, if you put MAKE_JOBS_NUMBER there, that's what's passed to make -j, so, in theory, yes, although it's extremely unlikely make will ever find 1000 jobs it can execute in parallel. There are dependencies during builds!

Your settings in poudriere.conf decide how many ports are built in parallel, with these settings, poudriere would attempt to create 1000 builder jails and use them in parallel (and each of them trying to use up to 1000 jobs).

All in all, these settings are extremely insane and will probably just crash your machine.
 
No, I just tested with the settings listed. It creates just one make process.
The setting ALLOW_MAKE_JOBS=YES does not work.
 
I wouldn't be surprised if there was a sanity check preventing such insane configurations.

Apart from that, check the port you're trying to build. If it has MAKE_JOBS_UNSAFE, there's nothing you can do.
 
All I can tell you is that this option works as advertized, I use it quite often and there was never a problem. You should find enough pointers in my answers, there's something wrong with your configuration for sure. And stop trying insane stuff.

The default configuration (ncpu builders, no make jobs) is somewhat conservative but will max out CPUs as long as all builders are in the "build" phase. Allowing make jobs for some selected packages will help if they typically end up building alone. If your goal is to always max out CPUs, something like reducing(!) the number of builders to 2 or 3, while allowing make-jobs for all packages, will do the trick. There's almost never a point in increasing either the number of builders OR the number of make jobs, both default to ncpu.
 
My experience is that with my 8=4x2-core CPU poudriere allows me to build 8 different ports in parallel.
But not one port using the 8-cores.
 
My experience is that with my 8=4x2-core CPU poudriere allows me to build 8 different ports in parallel.
But not one port using the 8-cores.
That's the default configuration. Setting ALLOW_MAKE_JOBS=yes will use 8 jobs for every port where that's possible in addition (ending up with up to 8*8=64 jobs when all 8 builders are in the build phase). If that doesn't happen on your machine, check all your configs. You will for example see everything poudriere puts in make.conf in the build log of a port.
 
I just looked it up on my build host. Try setting MAKE_JOBS_NUMBER= in /usr/local/etc/poudriere.d/make.conf.

I remember being confused as hell about poudriere's configuration, and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.

Still can't tell you what's happening there, but I think that's how it works on my machine.
 
and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.
These two have nothing to do with each other[*]. poudriere.conf is for configuring poudriere, it defines how the application itself operates. make.conf(5) is for setting generic build time options.

[*] Some options in poudriere.conf to define how ports are built, but only from a high level view. By default it will spawn as much jobs as you have cores (ncpu), each job will run on one core. ALLOW_MAKE_JOBS=YES removes that limitation and allow a job to use all cores. But as you normally have jobs running on the other cores this may just slow everything down. ALLOW_MAKE_JOBS_PACKAGES allows you to be more selective when to turn this on, you can add llvm* for example. If you have a lot of jobs waiting on a single core build of llvm10 for example and all the other jobs are idling.

Some ports ignore this setting completely and will only use one core regardless if you enabled ALLOW_MAKE_JOBS_PACKAGES or ALLOW_MAKE_JOBS.
 
I remember being confused as hell about poudriere's configuration, and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.
It's a feature. You can have for example jail- and ports-tree-specific make.conf entries with poudriere(8) (it's all documented in the manpage).
These two have nothing to do with each other. poudriere.conf is for configuring poudriere, it defines how the application itself operates. make.conf(5) is for setting generic build time options.
Sure, but MAKE_JOBS_NUMBER is a make.conf configuration. I see no reason to change it though…
 
So, /usr/local/etc/poudriere.d/make.conf (and its jail-/tree-specific cousins) configure what make does when buildings ports in poudriere's jails, except in the case of MAKE_JOBS_NUMBER=, which is ignored, unless ALLOW_MAKE_JOBS=yes is also set in /usr/local/etc/poudriere.conf, or the package being built is matched in ALLOW_MAKE_JOBS_PACKAGES="" in /usr/local/etc/poudriere.conf.

This behavior may be documented, and it may make sense after years of practice, but I call that a mess.
 
Some usage tips then perhaps.
  • Use ZFS, this is a no-brainer, if you have ZFS configure poudriere to use it. Creating a clean jail for a job is MUCH faster if it can just clone a jail.
  • Enable TMPFS. This will also speed up building as the intermediate files are stored on a RAM disk. My lowly Core i5 only has 16 GB and has no problems with this turned on. I can build even the biggest ports without issues.
  • Look at your build runs, are there ports that hold up the queue? Then add those to ALLOW_MAKE_JOBS_PACKAGES
  • Leave PARALLEL_JOBS and PREPARE_PARALLEL_JOBS alone, you usually don't need to change them.

Don't put any weird compiler options or other "optimizations" in /usr/local/etc/poudriere.d/make.conf. Use that file to set some defaults (DEFAULT_VERSIONS) or port options (OPTIONS_SET or OPTIONS_UNSET).
 
  • Thanks
Reactions: mtu
but I call that a mess.
And nobody cares.

(btw, the only thing that you could maybe call a "mess" is the fact that MAKE_JOBS_NUMBER isn't documented in any manpage. Of course it is documented in /usr/ports/Mk/bsd.port.mk, but that's not the most obvious place. OTOH, it's pretty obvious that overriding the number of jobs is an entirely different thing than enabling/disabling parallel builds.)
 
Some ports ignore this setting completely and will only use one core regardless if you enabled ALLOW_MAKE_JOBS_PACKAGES or ALLOW_MAKE_JOBS.

For some mysterious reason, ports such as rust and qt6-webengine decided to start ignoring the ALLOW_MAKE_JOBS=yes setting in poudriere.conf, even though they had never ignored it before. I have no idea what caused this mysterious change in the behavior of those ports.
 
Ports don't look at any poudriere configuration variables. Poudriere will pass appropriate ports variables (like MAKE_JOBS_NUMBER) to the port build. There are two typical issues with parallel building of software:

1. Some build systems are just broken when building in parallel. The typical reason is incomplete (internal) dependencies. When following a strict build order (single job), all is fine, but when multiple jobs are available, it can happen that the build system tries to build something before all prerequisites are built, which would then fail. This seems to be partially the issue with qt6-webengine, see below. It's also why the variable MAKE_JOBS_UNSAFE exists, which is meant to be put in a port's Makefile when the software is known to fail building in parallel ... it will force using only a single job no matter what for that port.

2. The traditional build system for software on *nix is make(1), which offers the -j flag to specify the number of parallel jobs – the names of the ports variables directly derive from that. Now more and more software uses different ("modern") build systems. FreeBSD's ports framework has support for several of them that are widely used (e.g. cmake, meson, ...) and knows how to tell them to use multiple jobs. Some other software uses "fully custom" build systems, so the port itself must figure out how to make them build in parallel. This seems to be the case for rust, see below. With all the varieties how to configure a build system for parallel jobs, problems are to be expected, either by issues with the tools used or errors in using the build system.

In lang/rust, we find this:
Bash:
do-build:
    @cd ${WRKSRC} && \
        ${SETENV} ${MAKE_ENV} ${PYTHON_CMD} x.py dist --jobs=${MAKE_JOBS_NUMBER}
So, this software uses a python script for its build, which offers a --jobs flag. The port passes the number of jobs wanted to this script. If this doesn't work any more, upstream introduced a bug (or a breaking change).

Looking at www/qt6-webengine:
Bash:
# The build system reads the environment variable $NINJA_PATH to decide whether
# to boostrap ninja or not (and also to invoke it afterwards). CC and CXX are
# read by some Chromium code to determine which compiler to invoke when running
# some configuration tests.
CONFIGURE_ENV+=    NINJAFLAGS="-j${MAKE_JOBS_NUMBER}" \
                   NINJA_PATH="${LOCALBASE}/bin/ninja"  \
                   PATH=${CONFIGURE_WRKSRC}/bin:${LOCALBASE}/bin:${PATH}
MAKE_ENV+=         CC="${CC}" CXX="${CXX}"                 \
                   C_INCLUDE_PATH=${LOCALBASE}/include     \
                   CPLUS_INCLUDE_PATH=${LOCALBASE}/include \
                   ${CONFIGURE_ENV}
# Avoid running multiple make(1) jobs, but only those.  Otherwise the build
# fails intermittently due race conditions if multiple ninja instances are
# running at the same time (mostly for the targets "WebEngineCore" and
# "convert_dict").
#
# MAKE_JOBS_UNSAFE is too invasive because it also affects the number of jobs
# for ninja(1) and would slow everything down which we don't want.  We pass the
# real number of make jobs via MAKE_JOBS_NUMBER to ninja(1) to CONFIGURE_ENV.
DO_MAKE_BUILD=    ${SETENV} ${MAKE_ENV} ${MAKE_CMD} ${MAKE_FLAGS} ${MAKEFILE} -j1 ${MAKE_ARGS:N${DESTDIRNAME}=*}
So, this suggests the build system there actually uses make(1), but delegating the actual build to ninja. For some reason, make must be forced to only use a single job, while ninja should use the value from NINJAFLAGS. If this suddenly doesn't work any more, upstream must have introduced some breaking changes to their build.

(code blocks are "make" syntax, seems the board doesn't support that and "bash" is similar enough for these snippets to give somewhat useful syntax highlighting)
 
Back
Top