Synth compiling latest libX11, but pkg upgrade -r Synth not installing it on my system?

rwv37

rwv37

When I do a "synth build topLevelPackages.txt", one of the things that Synth builds is libX11 (which is not one of those top level packages, but a dependency of some of them). But afterwards, when I do a "pkg upgrade -r Synth", Synth updates all the packages on my system except libX11; "pkg version" still lists my system's libX11 as an old, out of date one (1.6.9_2,1) rather than the latest (1.6.11,1).

This has happened for several update cycles now, and libX11 is always the only one that doesn't get upgraded. I had been guessing that some dependency which uses libX11 had some issue causing it to not accept versions of libX11 past 1.6.9_2,1. So up until now I've just let it slide, thinking it would be fixed in an upcoming update. But it's been a while, so tonight I decided to look into it further. One of the first things I did was check Synth's logs from the most recent update cycle. Everything succeeded, including libX11... but I was surprised to see that the version of libX11 that Synth built was 1.6.11,1.

Why might "pkg upgrade -r Synth" just leave my old 1.6.9_2,1 on my system, even though Synth built the new 1.6.11,1?

I thought maybe it's not being moved onto my system because it's just being built as a compile-time dependency, but "pkg autoremove" doesn't remove it (or anything else), so unless I'm misunderstanding, it's a runtime dependency.
 
Update: Just tried something:

(1) "pkg remove fsharp" (this was the only top-level package that was causing a need for libX11).
(2) "pkg autoremove" (removed a bunch of stuff, including libX11).
(3) "synth install lang/fsharp" (installed all that stuff that was just removed, including libX11).

But to my surprise and confusion, synth install installed the old libX11 1.6.9_2,1.

Double-checked /var/log/synth/x11___libX11.log. It says it built libX11 1.6.11,1 (earlier this week).
 
And another update:

I grepped all of synth's log files for 1.6.11, and it showed a bunch of hits in a bunch of files. But then I grepped for 1.6.9_2, and it got two and only two hits, both in the same file, devel___tbb.log. Here they are, along with a little context (and ignoring a bunch of intervening lines that seem unimportant to me, referring to other packages that doxygen, graphviz, cairo, and/or mesa-libs depend on):

Code: 
--------------------------------------------------------------------------------
--  Phase: build-depends
--------------------------------------------------------------------------------
===>   tbb-2020.3 depends on executable: doxygen - not found
===>   Installing existing package /packages/All/doxygen-1.8.15_3,2.txz
Installing doxygen-1.8.15_3,2...
`-- Installing graphviz-2.44.0_6...
|   `-- Installing cairo-1.16.0,2...
|   | `-- Installing mesa-libs-19.0.8_2...
|   |   `-- Installing libX11-1.6.9_2,1...
|   |   `-- Extracting libX11-1.6.9_2,1: .......... done

None of tbb, doxygen, graphviz, cairo, or mesa-libs are in my top level packages. Of them, only tbb is installed on my system, which is because it's required by par2cmdline-tbb, which is one of my top level packages (and it's the only one that requires tbb).

So I guess mesa-libs needs libX11 1.6.9_2,1 for some reason? And since my system needs libX11 (due to fsharp), pkg therefore wants to use 1.6.9_2,1, even though the one and only one package that requires 1.6.9_2,1 isn't installed or (apparently) needed on my system?

I don't really know enough about any of this stuff to understand why mesa-libs needs 1.6.9_2,1 (even assuming that guess is correct in the first place). I've looked in mesa-lib's Makefile, but nothing glaring has jumped out at me.

Any ideas on how to proceed would be appreciated. Thanks.
 
OK, I still don't understand what went wrong here, but in any case, I've fixed the symptom, at least.

(1) "synth rebuild-repository": This deleted a bunch of old versions of packages from /var/synth/live_packages/All, including libX11 1.6.9_2,1. That directory also contained a copy of 1.6.11,1, which "synth rebuild-repository" did not delete.

(2) "synth build topLevelPackages.txt": Built several packages, not including libX11.

(3) "pkg upgrade -r Synth": Upgraded one and only one package: libX11. Replaced the old 1.6.9_2,1 with the latest 1.6.11,1.
 
Something like this is happening again. Either what I think pkg should be doing is not what pkg should be doing, or else pkg isn't doing what it should be doing. I suspect the former, but in either case, I'd appreciate if someone could explain to me why what's happening is happening, and how I should get what I expect to happen to happen instead.

This time, when I do the "pkg upgrade -r Synth", pkg is upgrading all of my out-of-date ports except for py37-urllib3. Synth has compiled the latest version (1.25.10,1), but pkg is leaving my old version (1.25.7,1). And (unlike in the prior episode, where synth compiled the latest version of libX11 but also left the old one) the latest (1.25.10,1) seems to be the only version that synth knows about: All synth logs refer to 1.25.10,1, none to 1.25.7,1 or any other version, and the /var/synth/live_packages/All directory contains only version 1.25.10,1 not 1.25.7,1 or any other version.

So why might "pkg upgrade -r Synth" not upgrade py37-urllib3 with the version that synth has?
 
Are you following quarterly branches by any chances ?

The fact that pkg wouldn't upgrade a specific packages may indicate that some installed package were not rebuild inside synth build and therefore cannot be reinstalled from the synth build.
The strategy of pkg is to be very paranoid of dependency breakage, so the best ways to upgrade is to build every installed package (you can still use pre-fetch package), and then let pkg to upgrade all needed dependency
 
Thanks for the reply.

Synth did build the latest py37-liburl3, and it is sitting there in /var/synth/live_packages/All. No other version of it is present there, and no synth logs refer to any version besides it.

By "quarterly branches", you mean the base FreeBSD system? If so, no, I'm not. I have 12.1-RELEASE installed (since soon after it first came out), and have not updated it.

For ports, I upgrade everything fairly frequently (usually -- and I guess probably on average -- once a week). When I switched over to synth (a couple months ago?), having previously used portmaster, I did so as follows:

(1) Uninstalled all ports.
(2) Manually installed synth (via "make install"). This also caused pkg to be installed, if I remember correctly.
(3) Made a file containing a list of the packages I actually want (topLevelPackages.txt).
(4) Did a "synth build topLevelPackages.txt" and then got them (and their runtime dependencies) into my base system... I forget how, but certainly via pkg or synth. I forget the syntax... maybe something like "synth install topLevelPackages.txt"?

Since that initial installation I have been very consistent in using only synth and pkg, never anything like portmaster or direct use of make (well, "make config", but nothing else). The way I do so is as follows (and checking the results for errors, reading /usr/ports/UPDATING, etc.):

To upgrade my installed packages:

(1) portsnap fetch update
(2) synth status topLevelPackages.txt
(3) synth build topLevelPackages.txt
(4) pkg upgrade -r Synth

To install a new package:

(1) (upgrade my installed packages, as above)
(2) (add the new package to topLevelPackages.txt)
(3) synth status topLevelPackages.txt
(4) synth build topLevelPackages.txt
(5) synth install <new-package-name>

I believe the only times I've done anything different were when trying to diagnose and fix this problem, and even then I used only pkg and synth (such as is described in one of my earlier comments in this thread, where I used synth rebuild-repository). Oh, and I recently (but not before this problem with py37-urllib3 first occurred) added a cron job to do a "portsnap -I auto" and email me if any ports are out of date.
 
Wait, upon reading your message again, it occurred to me that (strictly speaking) it's not true that synth rebuilt the latest py37-urllib3. It's true that synth has built it, but looking at the file modification dates of its log (in /var/log/synth) and the package itself (in /var/synth/live_packages/All), it has not been built since last Friday (whereas I have attempted to upgrade several times since then).

So maybe the package's file modification time is causing pkg not to pick it up? But even if so, it should have picked it up the time that synth did compile it, no?
 
rwv37 said:
So maybe the package's file modification time is causing pkg not to pick it up? But even if so, it should have picked it up the time that synth did compile it, no?
Click to expand...

Well, so much for that theory: As of yesterday libX11 has another new version, so I once again upgraded my packages. This time pkg did grab the latest py-urllib3 from synth's repository... but synth still hasn't compiled it since last Friday. So whatever the issue is, apparently it's not caused by the file modification time.
 
For the quarterly branches I was referring to /etc/pkg/FreeBSD.conf that need to be copied to /usr/local/etc/pkg/repos/FreeBSD.conf and then change the copy to use latest instead of quarterly
Code: 
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

What is the diff between synth status topLevelPackages.txt and synth status ?
My hypothesis is that the missing package have a dependency on x11/libX11

rwv37 said:
To upgrade my installed packages:

(1) portsnap fetch update
(2) synth status topLevelPackages.txt
(3) synth build topLevelPackages.txt
(4) pkg upgrade -r Synth
Click to expand...

After the step 3 there is a need to synth rebuild-repository(this will rebuild the local synth repository) , and then if you change some default settings of pkg (like REPO_AUTOUPDATE) you have to pkg update before step 4
 
monwarez said:
For the quarterly branches I was referring to /etc/pkg/FreeBSD.conf that need to be copied to /usr/local/etc/pkg/repos/FreeBSD.conf and then change the copy to use latest instead of quarterly
Code: 
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
Click to expand...

I don't know anything about this. For what it's worth:

(1) I may be misunderstanding what "latest" and "quarterly" mean here, but I get package updates a whole lot more often than quarterly... often daily or even more frequently. It's been this way both before (long, long before) and after I changed my flow to use synth. Is it possible that this is a choice that I made once long ago as a part of the original FreeBSD installation? Like, just a checkbox in the install GUI?

(2) The /usr/local/etc/pkg/repos directory contains one and only one file: 00_synth.conf. I was not (directly) involved in its creation, nor in the fact that there's nothing else in that directory. Its contents are:

Code: 
# Automatically generated.

Synth: {
url : file:///var/synth/live_packages,
priority : 0,
enabled : yes,
}

monwarez said:
What is the diff between synth status topLevelPackages.txt and synth status ?
Click to expand...

Well, topLevelPackages.txt is just a file in which I've placed the names/origins of the packages that I actually want (in and of themselves). As I understand it (and as it sure seems to be) synth status <filename> causes synth to figure out what has to be built in order to get those packages up to date (including anything that they depend on). I think I've never used synth status (i.e. without a command line param), but as I understand it, it causes synth to figure out what has to be built in order to get all your currently installed packages up to date.

I think that in practice these work out to be the same thing (as long as you haven't done an end-around avoiding synth, e.g. make install blah). But in any case, I'm kind of doubting this is a synth issue... synth really has built the appropriate package, and really has made it it available in its repository. But pkg just... isn't taking it (or at least wasn't until last night).

I feel obligated to note here that whenever I say something like "as I understand it", that signals that I'm not even vaguely confident that I understand it :)

monwarez said:
My hypothesis is that the missing package have a dependency on x11/libX11
Click to expand...

If I'm understanding the output of pkg query correctly, I don't think so. It looks to me like py-urllib3 depends (directly or indirectly) on only these:

devel/gettext-runtime
devel/libffi
devel/py-asn1crypto
devel/py-cffi
devel/py-pycparser
devel/py-setuptools
devel/py-six
devel/readline
dns/py-idna
lang/python37
net/py-pysocks
print/indexinfo
security/py-certifi
security/py-cryptography
security/py-openssl

monwarez said:
After the step 3 there is a need to synth rebuild-repository(this will rebuild the local synth repository)
Click to expand...

As part of step 3, I do something that I believe is equivalent: synth build asks you if you want to rebuild the local repository; I say "y".

monwarez said:
and then if you change some default settings of pkg (like REPO_AUTOUPDATE) you have to pkg update before step 4
Click to expand...

I don't think I've ever changed default settings of pkg. If I have, it was either not recent or not intentional.
 
You must log in or register to reply here.
Back
Top