To be expected compile time of ports.

Alain De Vos

Alain De Vos

It would be interesting to know before-hand the to be expected compile time of ports on a "regular" PC.
Maybe i don't wait long enough. Maybe the port is broken ...
 
There will be quite a few factors ...

Number of cores, CPU speed, storage (NVMe versus SSD versus HDD), RAM, parallel make jobs, etc.

Might need to define "regular" PC and pick a few sample ports?
 
poudriere-devel-3.4.99.20240122_1

poudriere bulk -j main -J 3 -b latest -Ctv devel/electron27

HP ZBook 17 G2 (circa 2014), HDD, L2ARC (two USB thumb drives). Probed this morning: <https://bsd-hardware.info/?probe=8a5397997e>. Using only 3 of the 8 CPUs for poudriere, because I'm also using the ZBook for my day job.

I'll let you know the outcome.

In the meantime, a transcript is attached. Current progress:
Code: 
% poudriere status
SET PORTS   JAIL BUILD                STATUS         QUEUE BUILT FAIL SKIP IGNORE FETCH REMAIN TIME     LOGS
-   default main 2024-03-18_10h23m59s parallel_build    66    21    0    0      0    17     28 00:28:19 /usr/local/poudriere/data/logs/bulk/main-default/2024-03-18_10h23m59s
%

I imagine two or more updates to base becoming available (pkgbase) during the build, I'll probably choose to ignore the updates (to continue the run of poudriere).

When I woke the system at home, Konsole (with poudriere in one of its tabs) was no longer running. No explanation in /var/log/messages. Attached: a screenshot of poudriere after the event, I can't guess why Konsole disappeared but it seems that rust was building at the time.
 

Attachments

  • 2024-03-18 poudriere, rust.png
    2024-03-18 poudriere, rust.png
    97.1 KB · Views: 22
We might use some more or less random port that many use as a base line. So building llvm will be X*$Xorg, or something like that.
 
And would also affects whether ccache/sccache-overlay to be used or not.
And if used, how much chache size is given.

Note that sccache-overlay often behaves evil, with timeouts. But it would not the problem of sccache-overlay itlself, but poudriere[-devel]. Toooooo few timeouts for high loads are given, thus sccache server process cannot respond in-time. And much, much and much worse, I cannot find how to configure the timeouts.

Another point would be whether sysctl kern.maxvnodes is in proper range or not.
 
Totally system dependent. So many factors involved such as number of cores, clock speed disk I/O,etc, that no meaningful average could be achieved.

Bit like those "status" or loading bars you see in Windows that take 5 minutes to fill up to 99% then sit there for 25 minutes on the remaining 1%.

Thats why "spinners" exist.

It compiles, when it compiles...
 
That's why I would take a sufficient complex port for a baseline and norm against that. Something that most people who build ports on their own hardware will build anyway and that will be at the start of most build sessions. Like xorg, or something else. Only an idea...
 
blackhole said:
that take 5 minutes to fill up to 99% then sit there for 25 minutes on the remaining 1%
Click to expand...
Reminds me of that occasion where we, after it reached 100%, just let it continue to count up to 105% (while doing nothing).
Hard to believe how mad some people can get. Just joking...
 
I do remember the old joke about the guy who works on Windows copy dialogs on the phone with his wife, "I'll be home for dinner in 10 minutes..no, make that 5 minutes--no, make that two hours"
I also vaguely remember an old Linux from Scratch book where they took a package as the baseline, don't remember which, but very much like what Crivens suggests--take a common port and use it as your base.
 
Seriously depends on the processor... a Ryzen 5 1400 will take many hours to compile devel/llvm... a Ryzen 5 7600 will compile the same thing in about 2.5 hours... and a modern Threadripper will take even less time. Generally, if you have a lot of source code, like in case of LLVM or Chromium, expect an overnight job...

Well, if you manage to specify circular dependencies, your compilation will never finish 😈
 
Crivens said:
That's why I would take a sufficient complex port for a baseline and norm against that. Something that most people who build ports on their own hardware will build anyway and that will be at the start of most build sessions. Like xorg, or something else. Only an idea...
Click to expand...
And maybe recent lang/gcc* with LTO_BOOTSTRAP enabled?
They're quite CPU/Memory/tmp intensive.
lang/rust with default option is quete heavy, too.
 
I don't want to build rust or gcc just to know how long I might need to wait for what I need build. Knowing that my port to build takes 0.01 ${Rust} does not help me.
 
I used to run OpenBSD -stable, but when they switched to clang, they included llvm in base. I think it took about 20 to 30 minutes to compile the OS' base system and something like 3 hours to compile llvm. That was the end of that...

On the progress bars - that's why many OS boot splash screens now have spinning dots or some other moving objects rather than a progress bar. The progress bar is meaningless unless its "progress" is constant and predictable, if it stops two thirds of the way across for 10 minutes, then there is no point in having a progress bar at all.
 
cracauer@ said:
Fast computer with many cores are pretty cheap these days.
Click to expand...
Considering that a new Threadripper costs nearly $10k USD for just the processor... and compatible parts command a sizable premium, as well... :rolleyes: and new low-end stuff tends to match performance of premium stuff from about 10 years ago...
 
astyle said:
Considering that a new Threadripper costs nearly $10k USD for just the processor... and compatible parts command a sizable premium, as well... :rolleyes: and new low-end stuff tends to match performance of premium stuff from about 10 years ago...
Click to expand...

My dual Xeon E5-2697A v4 compiles Chrome in 90 minutes and costs much less than $1000 - with 384 GB RAM.

Likewise, AM4 systems can be had cheap now with as many cores as AM5 provides. Namely the 5950x with 16 full-speed cores, taking cheap DDR4 RAM. Also substantially less than $1000 and will probably do the job in 110-120 minutes if not faster.
 
cracauer@ said:
Likewise, AM4 systems can be had cheap now with as many cores as AM5 provides. Namely the 5950x with 16 full-speed cores, taking cheap DDR4 RAM. Also substantially less than $1000 and will probably do the job in 110-120 minutes if not faster.
Click to expand...
I have both AM4 and AM5 - and AM5 is faster to compile rust.
 
cracauer@ said:
I was talking price/performance. AM4 systems can be had cheap with a lot of memory and still have 16 cores.
Click to expand...
Yeah, but for same amount of money, you can have an AM5 system that may have supposedly lower specs, but very similar, and sometimes even better performance on the same task.
 
I love these xeon right before they came up with "scaleable". They're dead cheap now, they're fully industry-grade server tech, and they can scale up to 88 vcore already...
Alright, a contemporary chip might give twice the performance, but then you still need more than 16 real cores, and that is not cheap, and still not server tech.
 
Could it be that the memory bus is saturated well below these core counts? I'll check about the performance counters tomorrow, there should be a way to see memory wait times for the cores.
 
You must log in or register to reply here.
Back
Top