Poudriere and hyperthreading

K

kjpetrie

I'm currently in the middle of a bulk build waiting for devel/cmake-core to complete and wondering whether It's a mistake to have hyperthreading. I suspect without that a single package on which other builds are waiting might go twice as fast. Hyperthreading might be great for everyday use but for compiling, I wonder whether it's slowing things down as dependencies mean there's a limit to what can be done in parallel and it might be better to have more speed on fewer cores.

What do people think?
 
Using hyperthreading will absolutely speed things up for most tasks, definitely for C compilation.

As long as the increased number of parallel processes doesn't cause you to run out of RAM.
 
kjpetrie said:
I wonder whether it's slowing things down as dependencies mean there's a limit to what can be done in parallel and it might be better to have more speed on fewer cores.
Click to expand...
Play around with PARALLEL_JOBS, ALLOW_MAKE_JOBS/ALLOW_MAKE_JOBS_PACKAGES and USE_TMPFS. MUTUALLY_EXCLUSIVE_BUILD_PACKAGES can also be helpful, building 3 different versions of LLVM, a GCC and rust simultaneously is probably slower than building them one at time.

Having fewer concurrent jobs running will give the running jobs more room to use more cores for their own build. If you have plenty of RAM push more into tmpfs(4), RAM is usually faster than disk I/O. Find a good "sweet spot" to get the most bang for your buck.
 
The 'PARALELL_JOBS', 'ALLOW_MAKE_JOBS' and the 'ALLOW_MAKE_JOBS_PACKAGES' options for poudriere.conf might be what you are looking for.
That being said, if the actual build is run with a single or multiple threads is largely dictated by the port and the build chain it starts. Python/wheel or rust/cargo based builds are notoriously single-threaded and hence often a PITA holding up bulk buildjobs...
 
You must log in or register to reply here.
Back
Top