Ports with lots of dependencies got small upgrade.

Alain De Vos

Alain De Vos

One of the quarterly ports with lots of dependencies got a small upgrade. Now i must recompile 1700 ports. Rather annoying.
One would imagine quaterly is "stable", but in fact it's not.
 
I think you need to revisit the definition of stable. If the 1700 upgrades don't happens often and when it does happens it happens smoothly and in a planned manner, it is pretty stable for me.
And you are also compiling, if you upgraded using the binary system the upgrade process would be smaller.
 
I would consider it nice if ports with lots of dependencies only would get upgrades (including security upgrades) every two weeks.
I don't mind if ports with very few dependencies get upgrades in 6 hours.
For me the recompiling process becomes problematic.
 
This is exactly why I tried to start a Poudriere-based project to be able to recompile a specific list of ports that I choose (and leave everything else alone). Unfortunately, real life got in the way for me. But it's nice to know that the issue is still relevant, and needs to be addressed.
 
Do you really need to recompile 1700 ports? It's typically enough to recompile only what gets replaced by pkg upgrade. Current tooling is rather conservative and recompiles all dependencies, but you can cheat by building intermediate libraries explicitly.

Also there was an update on ports-mgmt/poudriere-devel recently, it should skip unnecessary recompiles.
 
Something is off here. A port with 1700 dependencies, when changed, only recompiles itself, not the dependencies.

You really need to be more verbose here. Which port?
 
Alain De Vos said:
I would consider it nice if ports with lots of dependencies only would get upgrades (including security upgrades) every two weeks.
Click to expand...
You do realize new quarterly branches are branched off main? So once every three months, when a new quarterly branch is created, a LOT of ports change. You have to catch up on three months of changes to main. A quarterly branch is kept stable during the three month period it is valid.

2024Q3 was just made, so it has 3 months worth of changes to main since 2024Q2.
 
Yeah, you can't avoid those changes either way.

And it is not quarterly's fault that you don't have enough RAM for the number of cores you use for compilation.
 
cracauer@ said:
Something is off here. A port with 1700 dependencies, when changed, only recompiles itself, not the dependencies.
Click to expand...
As noise mentioned on comment #6, if building ports with poudriere[-devel] without -S option, poudriere forced rebuilding ALL ports depending on updated ports.
And as mentioned, poudriere-devel starts ignoring such a overkill UNLESS LIBRARY VERSION IS BUMPED by default (and dropped -S option) starting from commit eef5ff69030c331452fea27a8cf18ab47f634789 of ports main branch.
Unfortunately, it was too late for 2024Q3 branch, and not yet MFC'ed.
And it has at least a rough edge. It IGNOREs unneeded rebuilds but lists as Queued inclusive.
Moreover, the check is NOT done for dry-run (except ports marked with IGNORE*= lines in its Makefile) and cannot know how much ports are rebuilt until actual poudriere bulk run.
 
when the resulting lib.so has the same version i dont update the ports that depend on it which rarely fails (api or structures change)
i even rename/copy the resulting .so when im pissed off or something / mostly never fails
this obviously breaks the deps db but...
 
cracauer@ said:
And it is not quarterly's fault that you don't have enough RAM for the number of cores you use for compilation.
Click to expand...

If it's from the chrome family it may not even respect the number of cores you set for the build. I think you have to cpuset() it to effectively restrict parallelism.

Yeah, glib is a nasty one, and electron has a direct dependency on it - no cheats for that, except to defer compilation until later. But glib has so many dependencies because our adaptation of it allows a lot of software from linux world to run on FreeBSD, unchanged. So it's worth it.
 
noise said:
If it's from the chrome family it may not even respect the number of cores you set for the build. I think you have to cpuset() it to effectively restrict parallelism.
Click to expand...

Limiting a parallel build of -j12 to 8 CPUs with cpuset would still require the RAM of 12 build processes.
 
Currently i do a ,
Code: 
/usr/bin/nice -n 15 /usr/local/bin/poudriere bulk
In the background with half the number of cores available.
So my desktop remains more or less responsive.
[Only Chromium i take from packages]
 
test.png
 
cracauer@ said:
Something is off here. A port with 1700 dependencies, when changed, only recompiles itself, not the dependencies.

You really need to be more verbose here. Which port?
Click to expand...
Not how I'm reading the Opening Post... A port would have ~20 dependencies - and 20 things that depend on it. There's a reason the ports tree is organized into leaf dependencies, trunk ports, and more. And dependency resolution has direction. Ever hear of Directed Acyclic Graphs? That really applies when trying to understand anything about dependency hell - which is the same thing in both Linux and BSD camps.
 
cracauer@ said:
Limiting a parallel build of -j12 to 8 CPUs with cpuset would still require the RAM of 12 build processes.
Click to expand...

Reasoning was the other way round. IIRC some part of the chrome build system scales up to the number of available cores, irrespective of the -j flags. Means if you have 12 cores but not enough RAM for 12 build processes, setting -j8 will not help. But cpuset may, depending on how the build system determines the number of cores available (didn't test, and don't want to).

Alain De Vos said:
Code:
/usr/bin/nice -n 15 /usr/local/bin/poudriere bulk
Click to expand...

I used to put the poudriere build into idle priority when building on my workstation, see idprio(). Doesn't help with other resources though (RAM, disk).
 
The FreeBSD Project page for ports could have been improved, instead it was removed.

Alain De Vos said:
… One would imagine quaterly is "stable", but in fact it's not.
Click to expand...

From the wiki:

A user experience (UX), for port and package installations and upgrades, that is more predictable and stable than the UX for the more dynamic main branch (familiarly: latest).

In essence, changes are limited to:
  • non-feature updates.

I guess, that could be improved to say "changes during each quarter".
 
You must log in or register to reply here.
Back
Top