Using ports-mgmt/poudriere with devel/ccache, tmpfs and parallel jobs can speed up your package build times. How much depends entirely on your hardware. Keep in mind there are quite a few pages out on the Internet about this so when I did my set up, I pulled from several. My aim for this HOWTO was to pull these together in one place so other folks who use ports-mgmt/poudriere to build ports can benefit from this.
From the Man page (ccache):
"ccache is a compiler cache. It speeds up recompilation by caching the result of previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++."
The implication here is that the FIRST time you run a ports-mgmt/poudriere build you won't notice any difference in speed using ccache. Subsequent builds will be faster.
Settings for ccache:
Assumptions/Procedure:
Ccache automatically creates its configuration as <ccachedir>/ccache.conf. The only modification I made to mine was to increase the cache size from max_size = 5.0G to
max_size = 10.0G, mainly because my build machine is not used for anything else and I have a lot of disk space.
I also added the following 2 options to /usr/local/etc/poudriere.d/make.conf although I am not sure the devel/ccache directory reference is really needed since it is also in /usr/local/etc/poudriere.conf.
That's it. On to the tmps configuration.
Settings for tmpfs:
Set up ports-mgmt/poudriere to use tmpfs. The setting you choose will depend on how much ram your build system has. The "all" setting uses more ram because the entire build is done in memory. I have 96GB of ram so chose "all".
In /usr/local/etc/poudriere.conf:
Other specific /usr/local/etc/poudriere.conf settings:
ccache:
Parallel build settings will depend on your hardware: number of CPU cores and amount of ram. The settings below are for 2 6-core Xeons (24 virtual cores) and 96GB of ram so adjust accordingly based on your hardware.
Parallel builds:
These settings are for my specific port requirements so adjust accordingly. The "*" by some ports is because the version is concatenated to the port name, e.g., llvm60.
Priority boost:
My build times with none of these configurations and building ~55 ports, which ripples out to ~700 ports including dependencies, went from around 11 hours to around 2-3 hours, even after running
I would like to thank vermaden for inspiring me to write this HOWTO and rigoletto@ and SirDice for some of the setting suggestions. I need to also thank kpa for the original ports-mgmt/poudriere HOWTO.
If anyone sees any mistakes or omissions in this HOWTO, please bring them to my attention so I can correct them.
From the Man page (ccache):
"ccache is a compiler cache. It speeds up recompilation by caching the result of previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++."
The implication here is that the FIRST time you run a ports-mgmt/poudriere build you won't notice any difference in speed using ccache. Subsequent builds will be faster.
Settings for ccache:
Assumptions/Procedure:
- Have a running ports-mgmt/poudriere installation (see thread Thread 38859 to set up ports-mgmt/poudriere)
- Install devel/ccache on the ports-mgmt/poudriere build machine
pkg install ccache
- Configure devel/ccache
- Add
to /usr/local/etc/poudriere.confCode:
CCACHE_DIR=/var/cache/ccache
- Add
Ccache automatically creates its configuration as <ccachedir>/ccache.conf. The only modification I made to mine was to increase the cache size from max_size = 5.0G to
max_size = 10.0G, mainly because my build machine is not used for anything else and I have a lot of disk space.
I also added the following 2 options to /usr/local/etc/poudriere.d/make.conf although I am not sure the devel/ccache directory reference is really needed since it is also in /usr/local/etc/poudriere.conf.
CCACHE_DIR=/var/cache/ccache
WITH_CCACHE_BUILD=yesThat's it. On to the tmps configuration.
Settings for tmpfs:
- Configure tmpfs per the FreeBSD Wiki except the /etc/fstab should look like
Code:
tmpfs /tmp tmpfs rw,mode=1777 0 0Set up ports-mgmt/poudriere to use tmpfs. The setting you choose will depend on how much ram your build system has. The "all" setting uses more ram because the entire build is done in memory. I have 96GB of ram so chose "all".
In /usr/local/etc/poudriere.conf:
Code:
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX
# data - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all - Run the entire build in memory, including builder jails.
# yes - Enables tmpfs(5) for wrkdir and data
# no - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS=allOther specific /usr/local/etc/poudriere.conf settings:
ccache:
Code:
# ccache support. Supply the path to your ccache cache directory.
# It will be mounted into the jail and be shared among all jails.
# It is recommended that extra ccache configuration be done with
# ccache -o rather than from the environment.
CCACHE_DIR=/var/cache/ccacheParallel build settings will depend on your hardware: number of CPU cores and amount of ram. The settings below are for 2 6-core Xeons (24 virtual cores) and 96GB of ram so adjust accordingly based on your hardware.
Parallel builds:
Code:
# By default poudriere uses hw.ncpu to determine the number of builders.
# You can override this default by changing PARALLEL_JOBS here, or
# by specifying the -J flag to bulk/testport.
#
# Example to define PARALLEL_JOBS to one single job
# PARALLEL_JOBS=1
PARALLEL_JOBS=6
# How many jobs should be used for preparing the build? These tend to
# be more IO bound and may be worth tweaking. Default: PARALLEL_JOBS * 1.25
# PREPARE_PARALLEL_JOBS=1
PREPARE_PARALLEL_JOBS=36These settings are for my specific port requirements so adjust accordingly. The "*" by some ports is because the version is concatenated to the port name, e.g., llvm60.
Priority boost:
Code:
# Define pkgname globs to boost priority for
# Default: none
PRIORITY_BOOST="llvm* rust chromium"My build times with none of these configurations and building ~55 ports, which ripples out to ~700 ports including dependencies, went from around 11 hours to around 2-3 hours, even after running
poudriere pkgclean -A -j JAILNAME.I would like to thank vermaden for inspiring me to write this HOWTO and rigoletto@ and SirDice for some of the setting suggestions. I need to also thank kpa for the original ports-mgmt/poudriere HOWTO.
If anyone sees any mistakes or omissions in this HOWTO, please bring them to my attention so I can correct them.
Last edited:
in a nutshell: idprio is the way to go here, but as in-kernel tasks are still scheduled with higher priority, there's still a chance to impair other tasks on the machine (to the point when they're reacting slow as hell). In my experience, having as many build tasks as there are cores/threads (the default with poudriere) will, when they're set to idprio, not have any noticeable effect, the machine can be used normally for other things.
