Potential ports-mgmt/synth misconfiguration?

M

mefizto

Greetings all,

in another thread I was whining that ports-mgmt/synth took a lot of time. As usually, it turned out that part of the problem was my idiocy, I forgot to turn on usage of tmpfs.

However, I turned option [N] Fetch prebuilt packages, which I understand as this option making the synth build only those ports, for which I have enabled/disabled certain options for and fetch the rest from the FreeBSD mirrors. However, looking at the logs, everything was being built.

I found an old thread that it is rather difficult to implement the prefetch properly per marion's reply here https://forums.freebsd.org/threads/...-prebuilt-packages-is-true.60164/#post-345717.

The post also states that: On the other hand, if you let it build everything the first time, the pain of building everything from source will be manageable on subsequent runs.
Click to expand...

Could somebody explain this to me? Since the packages do not have changed options, why rebuilding them "eases the pain of subsequent build"?

Kindest regards,

M
 
Suppose the following scenario:
Start with a 'clean' slate, you've built nothing before and want to build package for PortA. PortA 1.0 has a dependency on PortB 1.0. PortB gets build, followed by PortA. So far so good. Now you update the ports tree, PortB gets updated to 1.1, PortA is still the same version. The PortA 1.0 package that was previously built depends on version 1.0 package of PortB. But PortB has been updated and gets rebuild (deleting the 1.0 PortB package). The package of PortA still depends on the 1.0 PortB package, that just got deleted. In order to keep the dependency chain correct PortA must also be rebuild, so it can correctly depend on version 1.1 instead of 1.0 of PortB.

Does that make sense?
 
SirDice said:
Suppose the following scenario:
Start with a 'clean' slate, you've built nothing before and want to build package for PortA. PortA 1.0 has a dependency on PortB 1.0. PortB gets build, followed by PortA. So far so good. Now you update the ports tree, PortB gets updated to 1.1, PortA is still the same version. The PortA 1.0 package that was previously built depends on version 1.0 package of PortB. But PortB has been updated and gets rebuild (deleting the 1.0 PortB package). The package of PortA still depends on the 1.0 PortB package, that just got deleted. In order to keep the dependency chain correct PortA must also be rebuild, so it can correctly depend on version 1.1 instead of 1.0 of PortB.

Does that make sense?
Click to expand...
I'm not using ports-mgmt/synth (using ports-mgmt/poudriere-devel for stable/15 and ports-mgmt/pkg_replace for main), but dependants (anything depending upon something other) are not always keen to the version of the dependencies.

For example, in case Port A wants version 1 of library provided by Port B, and assuming Port B initially provided portb.so.1.0 and creates portsb.so.1 and portsb.so as symlinks pointing to portsb.so.1.0.

In this case, when Port B is upgraded to version 1.1 and provided library becomes portsb.so.1.1 and symlinks portsb.so.1 and portsb.so points to portsb.so.1.1, if dependencies of Port A to Port B is recorded using portsb.so.1 or portsb.so, no need to rebuild Port A unless the upgrade of Port B is backward incompatible.

But ports-mgmt/poudriere{-devel} forcibly rebuilds Port A by default (can avoid it with option -S, though). This is the safest but most painful behavior.

Note that the example above is for LIB_DEPENDS.

To be honest, things are sometimes far more complicated, but explaining further would be confusing here, at least currently.
 
Use devel/ccache it will speed up rebuild times. Also you need to have 64GB RAM or more to be able to build in parallel otherwise limit the number of builders and jobs to 1 or you will hit out of memory and will start to swap on disk which will slow your build. Some ports like RUST and LLVM requires around 20GB RAM and almost all ports now use them.
 
Hi SirDice,

thank you for the reply, it is very clear and even I can follow it.

Saying that, I still do not understand the assertion that building the dependencies, instead of downloading them "eases the pain of subsequent build". Following your scenario, if PortB gets updated to 1.1, and the ports tree is updated, the process ports-mgmt/synth is running senses that and offers rebuild of PortA. Now since the updated PortB has already been built as a package by the FreeBSD team, it could be downloaded, instead of recompiled.

Now, it is quite likely that I do not understand the concepts like you and T-Aoki.

Hi T-Aoki,

thank you for the reply, yes this dependency issue is very complex. I have selected the ports-mgmt/synth because I had used it in the past, and it seemed to handle all the dependencies aplomb.

Hi VladiBG,

thank you for your reply.

I did not mention the fact that I did in fact enabled the devel/ccache.

Regarding the RAM, it is what it is. In another thread cracauer@ recommended 700MB per core, which I have far exceeded.

Hi T-Daemon,

thank you for the reply.

I understand that the thread is old, but does the underlying problem, i.e., "It's possible that either the fetch is failing or it's getting a mismatch on the checksum (because the mirror is still copying files over, as it's not changed atomically (which is a big issue with the distribution model))" not still exist?

Thank you for posting the link, I have the same problem, namely "In particular each time I upgrade my ports it recompiles gcc, rust, llvm etc. (I don't need them, they are only used to compile other packages) whereas their compilation options are the default ones and the port version are the same as the FreeBSD repository version."

I do not understand jmarino's explanation, perhaps I am too stupid for this. Or, maybe I should switch to
ports-mgmt/poudriere as more people appear to use it, to have more help.

Hi Eric A. Borisch,

thank you for the reply.

The listed changes are very enticing. However, I am not smart enough to understand how to install it from the brief description therein.

Kindest regards,

M
 
You must log in or register to reply here.
Back
Top