pkg gurus: RUN_DEPENDS on specific version is ignored by pkg when updating the dependency, but dependency list is modified?

S

sko

I have several slave ports that need to stay in version lock-step with the master port, yet those slaves have their own specific version strings.
Since this relation is not handled automatically for master/slave ports, I add a version specific dependency via RUN_DEPENDS+= masterport==${VERSION} to all the slave ports, which correctly triggers a rebuild of all slaves if the master port version changes.

However, pkg only honors this dependency when installing the port/package the first time (i.e. the master port is installed too), but when the master port is updated, all slaves are ignored by pkg despite the hard dependeny to a specific version of the master pkg. The worst part: pkg not only doesn't reinstall the slave pkg, it silently updates the version string in the dependencies of the slave pkg.


Here's a demonstration of this behavior:

The master Makefile (which is included by all slave port Makefiles) adds this to all slave ports:
Code: 
.  if defined(ANGIE_PORTREVISION)
RUN_DEPENDS+=   angie==${ANGIE_VERSION}_${ANGIE_PORTREVISION}:www/angie
.  else
RUN_DEPENDS+=   angie==${ANGIE_VERSION}:www/angie
.  endif

Packages angie (version 1.11.2) and angie-module-geoip2 are installed, the module correctly shows the version-specific dependency:

Code: 
root@angie:~ # pkg info angie
angie-1.11.2
Name           : angie
Version        : 1.11.2
[...]
root@angie:~ # pkg info -d angie-module-geoip2
angie-module-geoip2-3.4_1,1:
        openssl35-3.5.5
        libmaxminddb-1.12.2
        angie-1.11.2
        libmaxminddb-1.12.2 (libmaxminddb.so.0)

Now the revision of www/angie is bumped, correctly triggering a rebuild of all modules by poudriere:
Code: 
[...]
[00:00:02] Deleting angie-1.11.2.pkg: new version: 1.11.2_1
[00:00:02] Deleting angie-module-image-filter-1.11.2.pkg: new version: 1.11.2_1
[00:00:02] Deleting angie-module-perl-1.11.2.pkg: new version: 1.11.2_1
[00:00:02] Deleting angie-module-subs-1.11.2.pkg: new version: 1.11.2_1
[00:00:02] Deleting angie-module-testcookie-1.11.2.pkg: new version: 1.11.2_1
[00:00:02] Deleting angie-module-xslt-1.11.2.pkg: new version: 1.11.2_1
[00:00:03] Deleting angie-module-auth-jwt-0.9.0_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-auth-spnego-1.1.3_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-auth-totp-1.1.0_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-brotli-1.0.0.r_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-cache-purge-2.5.5,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-dav-ext-3.0.0_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-echo-0.64_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-enhanced-memcached-0.3_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-eval-2016.06.10_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-geoip2-3.4_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-headers-more-0.39_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-jwt-3.4.4,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-keyval-0.3.0_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-lua-0.10.29_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-ndk-0.3.4_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-njs-0.9.5,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-postgres-1.0.r7_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-redis2-0.15_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-set-misc-0.3.4_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-upload-2.3.0_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-vod-1.33_1,1.pkg: missing dependency: angie-1.11.2
[00:00:03] Deleting angie-module-zstd-0.1.1_1,1.pkg: missing dependency: angie-1.11.2
[...]

However, the pkg upgrade afterwards ignores the version-specific dependency:
Code: 
root@angie:~ # pkg upgrade
Updating latest-servers repository catalogue...
[angie] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01    
[angie] Fetching data: 100%  293 KiB 300.5 k/s    00:01    
Processing entries: 100%
latest-servers repository update completed. 853 packages processed.
All repositories are up to date.
Checking for upgrades (1 candidates): 100%
Processing candidates (1 candidates): 100%
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        angie: 1.11.2 -> 1.11.2_1 [latest-servers]

Number of packages to be upgraded: 1

3 MiB to be downloaded.

Proceed with this action? [y/N]:

And without pkg reinstalling the port, it just bumps the dependency version, which will break the version-specific dependency (i.e. the module won't work) and even cover all traces of whats gone wrong:
Code: 
root@angie:~ # pkg info -d angie-module-geoip2
angie-module-geoip2-3.4_1,1:
        openssl35-3.5.5
        libmaxminddb-1.12.2
        angie-1.11.2_1
        libmaxminddb-1.12.2 (libmaxminddb.so.0)

I really doubt this is intentional, as this pretty much breaks RUN_DEPENDS and defeats its purpose.

chapter 5.10.9 of the porters handbook states:

5.10.9. Minimal Version of a Dependency​

A minimal version of a dependency can be specified in any *_DEPENDS except LIB_DEPENDS using this syntax:
p5-Spiffy>=0.26:devel/p5-Spiffy
The first field contains a dependent package name, which must match the entry in the package database, a comparison sign, and a package version.The dependency is satisfied if p5-Spiffy-0.26 or newer is installed on the machine.
Click to expand...

'a comparison sign' can be either '>=' or '==' (a quick grep shows that some python packages are also using such RUN_DEPENDS, e.g. textproc/py-tree-sitter-language-pack), so that's not the issue here. Plus: poudriere correctly honors this version dependency - it's only pkg that doesn't (and hides its traces by messing with the dependency list of the slave pkg)

Can someone more knowledgeable about the innards of pkg shed some light on this? Is this really intentional or is this a bug?
If it is intentional: how should one handle dependencies on specific/minimal versions for packages, if not via this mechanism documented in the PHB section quoted above?
 
If you need to reinstall a port/pkg, you will need to change the PORTREVISION.
This is what PORTREVISION is for.
How about setting DISTVERSION to ${MODULE_VERSION}.${MODULE_PORTREVISION} and PORTREVISION to angie's PORTREVISION?
Edit: I have never installed angie so I haven't looked into it properly.
Edit: If ANGIE_VERSION changes but MODULE_VERSION does not change, you will need to change MODULE_PORTREVISION.
Edit: Set DISTVERSION to ${MODULE_VERSION}.${ANGIE_PORTREVISION} and PORTREVISION to ${MODULE_PORTREVISION}.
 
Charlie_ said:
If you need to reinstall a port/pkg, you will need to change the PORTREVISION.
This is what PORTREVISION is for.
How about setting DISTVERSION to ${MODULE_VERSION}.${MODULE_PORTREVISION} and PORTREVISION to angie's PORTREVISION?
Click to expand...

I often see pkg reinstalling various packages with "(direct dependency removed/changed: *)" as a reason when running pkg upgrade - so how if not via *_DEPENDS can one trigger this behavior?

Since the modules have their own version numbers from upstream, using the same PORTREVISION on all the ports would be complete chaos - the PORTREVISION shall be returned to 0 if the DISTVERSION increases.
Also e.g. a patch requiring a REVISION bump in only a module port shouldn't trigger rebuilding *all* modules *and* the master, since only the module changed.

So if PORTREVISION is the only workaround to handle DEPENDS for packages, section 5.10.9 in the hanbook needs to be removed/changed, since version dependency with pkg simply does not work as advertised there.
 
Edited.
I don't think 5.10.9 needs to be edited.
Isn't it a misunderstanding that version includes revision?

Maintainers can manually change PORTREVISION if necessary.
Edit: ports/Tools/scripts/bump-revision.sh is available.
 
sko said:
I often see pkg reinstalling various packages with "(direct dependency removed/changed: *)" as a reason when running pkg upgrade - so how if not via *_DEPENDS can one trigger this behavior?
Click to expand...
Maybe library version changes in LIB_DEPENDS?
 
Charlie_ said:
Edited.
I don't think 5.10.9 needs to be edited.
Isn't it a misunderstanding that version includes revision?

Maintainers can manually change PORTREVISION if necessary.
Click to expand...
And add it if none.

A bit off-topic but if upgrade causes some issue and need reverting, thus, causing versions going backward, wouldn't cause pkg to be rebuilt.
PORTEPOCH is just for this case. Adding and/or incrementing it causes the whole bunch of version string to be "going forward".
And once PORTEPOCH is added, it SHALL not be decremented nor removed.
See Chapter 5.2.3.2 of the Handbook for details.
 
Charlie_ said:
Edited.
I don't think 5.10.9 needs to be edited.
Isn't it a misunderstanding that version includes revision?

Maintainers can manually change PORTREVISION if necessary.
Edit: ports/Tools/scripts/bump-revision.sh is available.
Click to expand...

It also doesn't work on version changes:

Code: 
# pkg info | grep angie
angie-1.11.1                   Efficient, powerful and scalable reverse proxy and web server
angie-module-geoip2-3.4_1,1    Angie GeoIP2 dynamic module                                                                  
angie-module-lua-0.10.29_1,1   Angie Lua dynamic module                                                                      
angie-module-ndk-0.3.4_1,1     Angie NDK dynamic module
angie-module-njs-0.9.5,1       Angie NJS dynamic module
angie-module-zstd-0.1.1_1,1    Angie zstd dynamic module
root@angie:~ # pkg info -d angie-module-geoip2                                                                              
angie-module-geoip2-3.4_1,1:                                                                                                
        openssl35-3.5.5                                                                                                      
        libmaxminddb-1.12.2                                                                                                  
        angie-1.11.1                                                                                                        
        libmaxminddb-1.12.2 (libmaxminddb.so.0)        
root@angie:~ # pkg upgrade                              
Updating latest-servers repository catalogue...                                                                              
[angie] Fetching meta.conf: 100%    179 B   0.2 k/s    00:01                                                                
[angie] Fetching data: 100%  293 KiB 300.2 k/s    00:01  
Processing entries: 100%                                  
latest-servers repository update completed. 853 packages processed.
All repositories are up to date.                        
Checking for upgrades (1 candidates): 100%                
Processing candidates (1 candidates): 100%                
The following 1 package(s) will be affected (of 0 checked):                                                                  
                                                             
Installed packages to be UPGRADED:          
        angie: 1.11.1 -> 1.11.2 [latest-servers]
                                                             
Number of packages to be upgraded: 1          
                                                             
3 MiB to be downloaded.                      
                                                             
Proceed with this action? [y/N]: y                          
[angie] [1/1] Fetching angie-1.11.2: 100%    3 MiB   2.9 M/s    00:01  
Checking integrity... done (0 conflicting)            
[angie] [1/1] Upgrading angie from 1.11.1 to 1.11.2...
[angie] [1/1] Extracting angie-1.11.2: 100%          
You may need to manually remove /usr/local/etc/angie/angie.conf if it is no longer needed.
You may need to manually remove /usr/local/etc/angie/prometheus_all.conf if it is no longer needed.
root@angie:~ # pkg info -d angie-module-geoip2
angie-module-geoip2-3.4_1,1:              
        openssl35-3.5.5                  
        libmaxminddb-1.12.2                                                                                                  
        angie-1.11.2                                    
        libmaxminddb-1.12.2 (libmaxminddb.so.0)

I still find it disturbing/wrong that pkg updates the dependency list without updating the package.


T-Aoki said:
Maybe library version changes in LIB_DEPENDS?
Click to expand...
IIRC those give the explicit reason "library changed"?


edit:
PORTEPOCH was necessary when moving to upstream versions for the modules. When I took maintainership of angie*, all modules inherited the version of the www/angie port, which made it impossible to distinguish the actual upstream version of the module without looking at the Makefile. Of course several modules had lower version numbers or used completely different versioning, so to make sure they all get reinstalled after that change, I introduced the PORTEPOCH.
 
If all the "slave" (should we call it "child" to shut up annoying non-technical persons who says "YOU SHALL NOT USE TERMS LILE MASTER OR SLAVE!!!!!"? I've encountered before:eek:) ports have exact same DISTVERSION (or PORTVERSION), PORTREVISION and PORTEPOCH (say, not setting independently in Makefile of slave ports) as the "master (parent?)" port, they should be automatically built by poudriere and upgraded by pkg.

Otherwise PORTREVISION bumps on slave ports should be considered as mandatory if slave ports themselves are NOT updated to follow changes in master port.
 
You must log in or register to reply here.
Back
Top