Poudriere, how to save time on compilation ?

RadicalEdhard

RadicalEdhard

Hi buddies, i'm trying to port SimulIDE and i have a a slight inconvenience with dependencies building. It's very, very long for some of them, like lang/rust. I've changed some settings, like the number of parallel tasks, but it's still very slow, so here's my question. Do you have any configurations or tips to save time on compilation when porting a new port? Have a nice day everybody !

edit: I've checked the source code and i don't understand why it's trying to build lang/rust. Simulde is based on Qt and written in C++.
 
Note i use ALLOW_MAKE_JOBS for building all my 1500 packages.
It gives good results & speed.
Only you must tune this for your hardware memory & disk space,
BUILDS=4
PARALLEL_JOBS=4
 
Finally i'm trying to build lang/rust 1.92 alone with only 2 jobs.
I'm trying to understand why my laptop freezes when editing links only on this port.
 
Building Rust using 2 threads will take a very long time. ( likely many hours ). If you have enabled LTO in the Build it will spend long time on the linking phase at the end of the build.
 
ALLOW_MAKE_JOBS=yes will use all cores , as long as BUILDS, PARALLEL_JOBS is bigger then 0.
BUILD & PARALLEL jobs is trial and error you start small and increase until it freezes.

In /etc/make.conf i have, MAKE_JOBS_NUMBER=9 , for 12 cores , this means 3 cores are always free for me to surf the internet.
 
RadicalEdhard said:
Hi buddies, i'm trying to port SimulIDE and i have a a slight inconvenience with dependencies building. It's very, very long for some of them, like lang/rust. I've changed some settings, like the number of parallel tasks, but it's still very slow, so here's my question. Do you have any configurations or tips to save time on compilation when porting a new port? Have a nice day everybody !

edit: I've checked the source code and i don't understand why it's trying to build lang/rust. Simulde is based on Qt and written in C++.
Click to expand...
It's likely building rust because QT and friends depends on mesa a few levels down.

To answer your question, you can use sccache/ccache but it's not 100% bulledproof (in short you may get weird fallouts)
You can use binary packages which may work fine on leaf ports but you may also end up some odd build issues.
The best answer is more computing power (and RAM) along with tuning.

Example, this is running on a Ryzen 7900 (12+12)

This tells Poudriere to run 7 workers (port builds) at the same time
Code: 
/usr/local/etc/poudriere.conf
...
PARALLEL_JOBS=7
...

This tells Poudriere to use 8 jobs (threads) per worker (port build)
Code: 
/usr/local/etc/poudriere.d/make.conf
DISABLE_VULNERABILITIES=yes
MAKE_JOBS_NUMBER_LIMIT=8

Optional but it helps iron out build and C/CXX/FLAGS issues, this needs to be adjusted to what hardware you're running on and the name of your jail
Code: 
/usr/local/etc/poudriere.d/143amd64-dev-make.conf
CPUTYPE?=znver4

You always want to overbook CPU capacity, there's no easy recipe but you need to balance it between amount of cores and possibly take amount of RAM into account
A good rule of thumb is that you want jobs to finish fast rather than doing loads in parallel

Things to take into account,
Far from all ports utilize the max amount of jobs allowed all the time. We spend a ton of time running configure scripts (autotools) which are very slow and compute intensive (one of many reasons to move away from it) compared to pretty much anything else such as cmake, meson, waf so you may end up with multiple workers running configure or another single threaded operation while you have maybe 2 doing compiling. If you have an older box but still many cores you might be better off dailing down PARALLEL_JOBS and up MAKE_JOBS_NUMBER_LIMIT but at the same time you don't want unnecessary idling cores but you'll inevitably run into it in some scenarios such building llvm, gcc or rust which some commonly used libraries depends on.
 
cracauer@ said:
Why are you building Rust and don't install the pkg?

If this is a new port you wrote, how come there are surprise dependencies?
Click to expand...
well, when i specify to retrive the rust binary rather than build it, it stubbornly insists on doing so. i think. I think it's because I'm on the Quarterly branch which contains Rust version 1.89, but Simulilde wants 1.92 which appears to be the latest. Now i'm surpised about this dependency because nothing in the source archive seems to use rust. All is written in C++ and built on Qt. The readme don't spicify something else than qt packages in the build dependecies. Is it possible that Poudriere is stubbornly building packages that are not part of the port's dependencies that we are trying to compile?
 
Well in fact you can very well quarterly packages and ports from quarterly.
I've been doing this for quite some time.
But there is a golden rule , or 99% packages or 99% ports, but if you go for 50% then your for problems.
 
cracauer@ said:
Yeah, you can't mix quarterly packages and building from ports.
Click to expand...
If you switch poudriere.conf to quarterly.
I am also using poudrire-dsh2dsh

I just did my xorg-minimal and xterm in 3 mins flat. No building at all.

Code: 
[00:02:59] [main-default] [2026-01-17_20h24m36s] [committing] Time: 00:02:57
           Queued: 191 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched: 191 Remaining: 0
[00:02:59] Logs: /poudriere/data/logs/bulk/main-default/2026-01-17_20h24m36s
[00:02:59] Cleaning up
[00:02:59] Unmounting file systems
Fetched the whole lot.
Screw you llvm.
 
Here is my setup.
My host Package System is running default quarterly branch.

pkg install -y pkg git-lite poudriere-dsh2dsh
git clone -b releng/14.3 https://git.freebsd.org/src.git /usr/src
git clone -b 2026Q1 https://git.freebsd.org/ports.git /usr/ports

poudriere jail -c -j main -v 14.3-RELEASE -K GENERIC -b -J8 -m src=/usr/src
poudriere ports -c -p default -m null -M /usr/ports
mkdir /usr/ports/distfiles
poudriere bulk -j main -p default -f pkglist -b quarterly

I change BASEFS to /poudriere. This is just a personal preference.
This is the only section I mess with for results. Change two things:
/usr/local/etc/poudriere.conf
Code: 
# Set to always attempt to fetch packages or dependencies before building.
# XXX: This is subject to change
# Default: off; requires -b <branch> for bulk or testport.
PACKAGE_FETCH_BRANCH=quarterly
# The branch will be appended to the URL:
#PACKAGE_FETCH_URL=pkg+http://pkg.FreeBSD.org/\${ABI}
# Packages which should never be fetched.  This is useful for ports that
# you have local patches for as otherwise the patches would be ignored if
# a remote package is used instead.
#PACKAGE_FETCH_BLACKLIST=""
# Alternatively a whitelist can be created to only allow specific packages to
# be fetched.
# Default: everything
#PACKAGE_FETCH_WHITELIST="gcc* rust llvm*"
PACKAGE_FETCH_WHITELIST="*"
So I uncomment PACKAGE_FETCH_BRANCH and change to quarterly.
Change whitelist to all. This should be default setting but I reinforce it.

The only thing that needs building during bulk is pkg itself. That is on first run. After that no building whatsoever.
 
cat poudriere.conf | grep -v ^\# | awk '!/^$/'
Code: 
ZPOOL=SSD
ZROOTFS=/poudriere
FREEBSD_HOST=https://download.FreeBSD.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
USE_PORTLINT=no
USE_TMPFS=all
DISTFILES_CACHE=/usr/ports/distfiles
BUILDS=2
PARALLEL_JOBS=2
PREPARE_PARALLEL_JOBS=4
NOLINUX=yes
ALLOW_MAKE_JOBS=yes
ALLOW_MAKE_JOBS_PACKAGES="llvm* gcc* rust* waterfox firefox-esr"
TIMESTAMP_LOGS=yes
BUILDER_HOSTNAME=myfreebsd
USE_COLORS=yes
URL_BASE=http://127.0.0.1/poudriere/
PACKAGE_FETCH_BRANCH=quarterly

cat poudriere_run
Code: 
export DISABLE_LICENSES=yes
export BATCH=yes
export myjail=pjailA
export myport=defaultA
export mybranch=2026Q1
export mypackagelist=./packagelist.txt
export myurl=https://git.freebsd.org/ports.git
poudriere ports   -u                     -p $myport -m git+https -B $mybranch -U $myurl
sleep 10
poudriere bulk -b quarterly  -j $myjail -p $myport -f $mypackagelist

cat poudriere_nice | grep -v ^\# | awk '!/^$/'
Code: 
nice -n 31 idprio 31 ./poudriere_run
pkg update -f
pkg upgrade
 
Alain De Vos said:
USE_TMPFS=all DISTFILES_CACHE=/usr/ports/distfiles BUILDS=2 PARALLEL_JOBS=2 PREPARE_PARALLEL_JOBS=4 NOLINUX=yes ALLOW_MAKE_JOBS=yes ALLOW_MAKE_JOBS_PACKAGES="llvm* gcc* rust* waterfox firefox-esr"
Click to expand...
Yes but none of that is necessary if you just do packages.
If you have Port Options you have no choice but building. Then those settings really help.
But for most people they are not changing options on rust or llvm. So let them download the packages. Those settings are not needed on normal programs. Only hogs.
 
Ooh but i have.
git clone llvm-project
git branch
* release/21.x
x@myfreebsd:~/git/llvm-project $ cat *.txt
Code: 
set -eu
cmake -G "Unix Makefiles" \
  -DLLVM_ENABLE_PROJECTS="clang;lld;clang-tools-extra;compiler-rt" \
  -DLLVM_TARGETS_TO_BUILD="all" \
  -DLLVM_INSTALL_UTILS=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr/local/llvm21 \
  -DLLVM_USE_LINKER=lld \
  ../llvm
cmake --build . -j 9
cmake --install .
 
Alain De Vos said:
Ooh but i have.

x@myfreebsd:~/git/llvm-project $ cat *.txt
Click to expand...
Die hards, take git repos.

git clone llvm-project
git branch
* release/21.x
Code: 
set -eu
cmake -G "Unix Makefiles" \
  -DLLVM_ENABLE_PROJECTS="clang;lld;clang-tools-extra;compiler-rt" \
  -DLLVM_TARGETS_TO_BUILD="all" \
  -DLLVM_INSTALL_UTILS=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr/local/llvm21 \
  -DLLVM_USE_LINKER=lld \
  ../llvm
cmake --build . -j 9
cmake --install .
:)
 
Phishfry said:
Yes but none of that is necessary if you just do packages.
If you have Port Options you have no choice but building. Then those settings really help.
But for most people they are not changing options on rust or llvm. So let them download the packages. Those settings are not needed on normal programs. Only hogs.
Click to expand...
People always tend to push their limits, this makes fun in life.
Eg, not to brag,
https://forums.freebsd.org/threads/...e-ruby-python-d-crystal-scala-go-odin.101328/
 
You must log in or register to reply here.
Back
Top