Synth: Introducing new custom package repository builder for FreeBSD and DragonFly

Yes, it happens on 2 different machines. Just to be sure I rebooted one, toggle NLS option ON - success. Toggle it OFF - failure as I described.

Let me know what else I can provide.
 
Hmm, this port doesn't build on DragonFly with NLS off due to broken dependencies.

Code: 
checking for xgettext... no
checking for msgmerge... no
checking for msgfmt... no
checking for gmsgfmt... no
configure: error: GNU gettext tools not found; required for intltool
===>  Script "configure" failed unexpectedly.
Please report the problem to gnome@FreeBSD.org [maintainer] and attach the
"/construction/news/pan/pan-0.140/config.log" including the output of the
failure of your make command. Also, it might be a good idea to provide an
overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

Stop.
make[1]: stopped in /xports/news/pan

what does your synth/logs/news___pan.log show? You haven't mentioned it yet.
 
The port fails at configure phase:

Code: 
===>  pan-0.140 depends on shared library: libgtk-x11-2.0.so - found (/usr/local/lib/libgtk-x11-2.0.so)
===>  Returning to build of pan-0.140
===>  pan-0.140 depends on shared library: libpango-1.0.so - found (/usr/local/lib/libpango-1.0.so)
===========================================================================

========================< phase : configure  >========================
===>  Configuring for pan-0.140
configure: loading site script /xports/Templates/config.site
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... (cached) /bin/mkdir -p
checking for gawk... (cached) /usr/bin/awk
checking whether gmake sets $(MAKE)... yes
checking whether gmake supports nested variables... yes
checking whether UID '0' is supported by ustar format... yes
checking whether GID '0' is supported by ustar format... yes
checking how to create a ustar tar archive... (cached) /usr/bin/tar
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether gmake supports nested variables... (cached) yes
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++48 accepts -g... yes
checking for style of include used by gmake... GNU
checking dependency style of g++48... gcc3
checking for gcc... gcc48
checking whether we are using the GNU C compiler... yes
checking whether gcc48 accepts -g... yes
checking for gcc48 option to accept ISO C89... none needed
checking whether gcc48 understands -c and -o together... yes
checking dependency style of gcc48... gcc3
checking how to run the C preprocessor... cpp48
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for egrep... (cached) /usr/bin/egrep
checking for ANSI C header files... (cached) yes
checking whether time.h and sys/time.h may both be included... yes
checking for localtime_r... yes
checking for close... yes
checking for tr1/unordered_set... yes
checking whether the compiler implements namespaces... yes
checking whether the compiler has ext/hash_set... yes
checking for gawk... (cached) /usr/bin/awk
checking whether gmake sets $(MAKE)... (cached) yes
checking for ranlib... /usr/local/bin/ranlib
checking for sys/types.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for memory.h... (cached) yes
checking for strings.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for stdint.h... (cached) yes
checking for unistd.h... (cached) yes
checking for errno.h... (cached) yes
checking for fcntl.h... (cached) yes
checking whether NLS is requested... no
checking for intltool >= 0.40.6... 0.51.0 found
checking for intltool-update... /usr/local/bin/intltool-update
checking for intltool-merge... /usr/local/bin/intltool-merge
checking for intltool-extract... /usr/local/bin/intltool-extract
checking for xgettext... no
checking for msgmerge... no
checking for msgfmt... no
checking for gmsgfmt... no
configure: error: GNU gettext tools not found; required for intltool
===>  Script "configure" failed unexpectedly.
Please report the problem to gnome@FreeBSD.org [maintainer] and attach the
"/construction/xports/news/pan/work/pan-0.140/config.log" including the output
of the failure of your make command. Also, it might be a good idea to provide
an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

Stop.
make: stopped in /xports/news/pan
===========================================================================
 
Hi marino@.
Got another question on synth: just how does one actually configure "external repository"?
In my file /usr/local/etc/pkg/repos/00_synth.conf the local repo /var/synth/live_packages is listed as repo. Will synth then fetch prebuilt packages from that repo if I configure to fetch prebuilt packages?

Thank you.
 
free-and-bsd@
pkg -vv
You should see two repositories enabled: FreeBSD and Synth

prefetching pulls from FreeBSD repository.

pkg -vv | grep CONS

Make sure CONSERVATIVE_UPGRADE is set to false (adjust pkg.conf as necessary to accomplish this)
 
marino@ said:
free-and-bsd@
pkg -vv
You should see two repositories enabled: FreeBSD and Synth

prefetching pulls from FreeBSD repository.

pkg -vv | grep CONS

Make sure CONSERVATIVE_UPGRADE is set to false (adjust pkg.conf as necessary to accomplish this)
Click to expand...
OK thank you very much. I've checked pkg.conf(5)() manual page for options, too. But the thing is I don't want packages from FreeBSD repo because I've had problems with them crashing for no obvious reason while the locally built one worked fine.
(to moderator: I have no idea why it attaches "()" after pkg.conf(5))
 
free-and-bsd said:
But the thing is I don't want packages from FreeBSD repo because I've had problems with them crashing for no obvious reason while the locally built one worked fine.
Click to expand...

Then leave prefetching option at the default, which is "don't". Synth will never fetch in that case.
With CONSERVATIVE_UPGRADE=false, freebsd repo would only be used if a required package isn't in synth repository.
You might even want to deinstall everything and reinstall from synth repository (explicitly with -r Synth to be sure), then you'll know everything is locally built (only valid if there were already packages installed on the system)
 
Actually, I've checked my configuration and the only enabled repo is that of live_packages. And I've already replaced all the local packages with those built with synth.
I was asking because, while building a new port, synth started to build packages already built, to my best knowledge. Meantime I don't remember updating the ports tree after the last build... which means nothing, of course. But then I also wanted to get once and for all clear about this interesting option of fetching packages. In portmaster that didn't seem to work nor to be going to work.
 
I think I found a bug here. Synth complains about a circular dependency when I try to override the default gcc setting in make.conf.

Code: 
doas synth status
Querying system about current package installations.
Stand by, comparing installed packages against the ports tree.

ports-mgmt/pkg scan aborted because a circular dependency on devel/binutils was detected.
... backtrace devel/gmake
... backtrace devel/binutils
... backtrace ports-mgmt/pkg
Unexpected pkg(8) scan failure!
Unfortunately, the system upgrade failed

Contents of /usr/local/etc/synth/*make.conf

Code: 
DEFAULT_VERSIONS+=ssl=libressl
OPTIONS_UNSET=BONJOUR AVAHI PGSQL MYSQL DEBUG V4L CUPS DOCS EXAMPLES HAL IPV6 NLS TEST PULSEAUDIO
OPTIONS_SET=DEVD
VIDEO_DRIVER=nvidia
ftp_curl_UNSET=  TLS_SRP
USE_GCC?=6

It works fine without "USE_GCC?=6". However I don't intend to use an ancient gcc compiler.

talsamon said:
Poudriere does not recompile a package if it does not change, Synth does (e.g webkit-gtk is recompiled every second run).
Click to expand...

I have the same problem. Whenever a library gets a slight bump, Synth recompiles ALL packages that depend on that library. Is this intended or just accidental, as it causes MAJOR recompiles everytime. For instance I've just recompiled freetype2 to enable LCD_FILTERUNG and it recompiled > 100 packages, including big ones like Firefox and Chromium (and worse llvm37). It kind of defeats the point of using shared libraries.
 
Try to reproduce that outside Synth, it's highly likely that the bug is in the ports system and not in Synth at all.
 
kpa said:
Try to reproduce that outside Synth, it's highly likely that the bug is in the ports system and not in Synth at all.
Click to expand...

I just tested this without Synth and it works fine with USE_GCC?=6 enabled.

Code: 
doas synth install net-im/telegram-purple

ports-mgmt/pkg scan aborted because a circular dependency on devel/binutils was detected.
... backtrace devel/gmake
... backtrace devel/binutils
... backtrace ports-mgmt/pkg
Unexpected pkg(8) scan failure!

cd /usr/ports/net-im/telegram-purple && make install clean works as intended.
 
It's not a bug. The circular dependency is *detected*. You caused it with "USE_GCC?=6" which says "use GCC for every port". GCC 6 has many dependencies, so every port has the same dependencies. The detection is valid.

I'm not addressing the point of "shared libraries" because it reflects a fundamental misunderstanding of how repository builders like poudriere and synth work. TDLR; it's not "accidental".
 
You need to figure out the proper way to change ports compilers, if there even is one. setting USE_GCC in make.conf is not valid. You could start with Mk/bsd.defaults.mk and change GCC_DEFAULT but that only affects ports already using GCC I suspect.
 
I'm not addressing the point of "shared libraries" because it reflects a fundamental misunderstanding of how repository builders like poudriere and synth work. TDLR; it's not "accidental".
Click to expand...

Good to know, thanks. Does poudriere exhibit the same behavior, e.g. rebuilding "everything" when a library changes slightly?

change GCC_DEFAULT but that only affects ports already using GCC I suspect.
Click to expand...

This is actually what I want. I'm fine with using clang for ports that support it but I want to use a different GCC version for all ports that depend on GCC.
 
archfan said:
Good to know, thanks. Does poudriere exhibit the same behavior, e.g. rebuilding "everything" when a library changes slightly?
Click to expand...

They have exactly the same behavior, but it's not based on "library changes". library versions, symbols, contents have nothing to do with it. The thing triggering a rebuild could be 3-4 levels removed.

This is actually what I want. I'm fine with using clang for ports that support it but I want to use a different GCC version for all ports that depend on GCC.
Click to expand...

Nothing beyond the standard 4.8 is guaranteed to work. Most ports will but not all. If even 4.9 would fully work, the default would be GCC 4.9.
 
I've used Synth a few times and it's pretty great for building in parallel.

However, I'm still confused about exactly how it goes about doing so, why it does certain things, and where I might be going wrong.

I've got 166 ports installed. I used to have lang/php56 but now I use lang/php70.

However, when I ran synth status it told me that it was going to rebuild lang/php56 -- but pkg doesn't report that it is installed.

So I checked dependencies and reverse dependencies as well -- no mention of lang/php56.

So I thought to run synth rebuild-repository but it STILL says it's going to build lang/php56 (which conflicts with lang/php70, which breaks stuff). Plus instead of rebuilding, it's going to build everything "N" New.

I want to use synth, but either I've found a bug in the ports tree, or there's something weird on my system, or there's another issue.

What do I need to post here that would help? There's nothing in /var/log/synth/* that was output recently.

I suspect that databases/adodb5 is to blame, as it requires php56, though adodb.org states it supports PHP7 (and I've confirmed that it does). I've reached out to the maintainers.
 
ooglek@, the default version of php is 5.6. see Mk/bsd.default-versions.mk
Unless you specifically declare another version in <profile>-make.conf, synth in going to take the defaults. (no, it doesn't matter what's in /etc/make.conf, that's not used. You need to define a new *make.conf per profile).
 
marino@, Thank you! That seems to have resolved the php56 issue for me.

When I run synth rebuild-repository why does the indicator of N (new), R (rebuild) and U (upgrade) all turn to New, when the versions installed haven't changed? Prior to running that, the N/R/U indicators seemed accurate and helped give me a good sense of what synth was going to do.

Also, like a recent thread, pkg audit -F, which pulls the latest vulnerability list from FreeBSD, came as a surprise to many, including me. For 90% of system admins, beyond synth status and synth upgrade-system, are there commands that one needs to run regularly or occasionally in the course of time passing?

Feature Request: a single synth command that runs make -C /usr/ports/category/port config on all of the out-of-date configurations.
 
ooglek said:
When I run synth rebuild-repository why does the indicator of N (new), R (rebuild) and U (upgrade) all turn to New, when the versions installed haven't changed? Prior to running that, the N/R/U indicators seemed accurate and helped give me a good sense of what synth was going to do.
Click to expand...

Because they are different ports. php 70 version of something is a different port than php 56 version of the same thing.

Also, like a recent thread, pkg audit -F, which pulls the latest vulnerability list from FreeBSD, came as a surprise to many, including me. For 90% of system admins, beyond synth status and synth upgrade-system, are there commands that one needs to run regularly or occasionally in the course of time passing?
Click to expand...

I'm not sure why you're asking me this. Synth just keeps a local repository up-to-date. You can utilitize that repository with pkg commands (maybe with -r Synth) or you can get Synth to manipulate pkg with commands like synth install, but I consider pkg audit out of scope. Synth will build the latest available, regardless of it's vulnerability state.

Feature Request: a single synth command that runs make -C /usr/ports/category/port config on all of the out-of-date configurations.
Click to expand...

With synth, you want the least number of configs. If the saved config would be the same as the defaults, you don't want it at all. You only want configs that differ from the defaults. There is even a script to help remove redundant configurations at Tools/scripts/redundant-opt-files.sh. The vast majority of out-of-date configuration would be removed, not re-saved. It's a one-time effort that needs to be done when moving to Synth which removes all the cruft caused by building live or with portmaster. So such a command is counter-productive.
 
Hello all.
Sorry if this is not the place to ask but I have a question regarding the use of synth.

I need two different versions of packages, a php70 version and a php56 version.
I created a profile through synth configure named php70 and one named php56
I created a file /usr/local/etc/synth/php70-make.conf
I created a file /usr/local/etc/synth/php56-make.conf

I also created a file /usr/local/etc/synth/php70-build.list
I also created a file /usr/local/etc/synth/php56-build.list
The build file looks like this
Code: 
www/apache24
lang/php70
lang/php70-extensions

My ports options directory is set to
[D] Port options directory /var/db/synth/ports-php70
if I do synth build /usr/local/etc/synth/php70-build.list It builds all the ports, but I do not get the ports options screens. I need to adjust some settings!
How do I get these?
Or must I do a make -C /usr/ports/category/port and copy the options from /var/db/ports/ to /var/db/synth/ports-php70?

If I want to build for the profile php56 I do it now by going to the configure screen and select php56.
Can I use a command line option to select the profile?

Thanks for your time.

regards
 
There are two ways of doing it.
  1. You can set and unset individual port options in phpXX-make.conf
  2. Option [D] under configure is where the port options files are. You might have different ones for each profile, they might be shared, and they might be in common with the system version.
What you could do is "cd <category/portname> && make config" and then copy /var/db/ports/<category>_<portname> over to the profiles's ports options directory (assuming you don't just use /var/db/ports)
 
What can I do about this. This is the reason for my recent troubles (amount other errors I suspect). The thing is that mysql56-client is not installed at the moment.

Code: 
synth status
Querying system about current package installations.
Stand by, comparing installed packages against the ports tree.

devel/subversion scan aborted because a circular dependency on databases/mysql56-client was detected.
... backtrace security/cyrus-sasl2
... backtrace net/openldap24-sasl-client
... backtrace security/heimdal
... backtrace ftp/curl
... backtrace devel/cmake
... backtrace databases/mysql56-client
... backtrace devel/apr1
... backtrace devel/subversion
Unfortunately, the system upgrade failed.
 
I got this error. Not sure why that is.

Code: 
pkg update
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating Synth repository catalogue...
pkg: file:///var/synth/live_packages/meta.txz: No such file or directory
repository Synth has no meta file, using default settings
pkg: file:///var/synth/live_packages/packagesite.txz: No such file or directory
Unable to update repository Synth
 
jonfr@, regarding the circular dependency, this is surely caused by non-compatible options that have been selected in the past.
Unless you know for a fact which ports require modifying, you might want to clear out your options directory (e.g. /var/db/ports).
The devel/apr1 might have the MySQL option selected for example.
Basically a circular dependency between ports A and B mean that both of them need the other to build. It doesn't happen with default options but easily could with non-default options.

secondly, are you using synth version v1.65?
in any case, remove /usr/local/etc/pkg/repos/00_synth.conf to clear those second set of errors.
 
You must log in or register to reply here.
Back
Top