Build port but install dependencies with pkg

Z

ztp

I'm sometimes finding that I want to rebuild a port to enable a single option, like DEBUG, for example. Otherwise I generally use standard packages for everything else. For some ports with a lot of dependencies, I want to avoid having to build all those dependencies from source. The last time I had to to do this I crafted a one-liner to take the result of # make all-dependencies-list and modify that to make the packages arguments to # pkg install -y.

I wound up doing something like this from that port's directory:

# make all-depends-list | cut -c 12- | xargs pkg install -y

which cuts off the /usr/ports/ from the beginning of each line. Alternately you can use build-depends-list or run-depends-list.

This all seems a bit janky. Is there an established way of doing this that I am missing?
 
In short, AFAIK, no. At least there is no supported way to do this that I know of. The reason being is the official package repository and the ports tree are not synced to the same ports tree revision at this time, therefore creating a case where the dependencies may or may not be in sync with the port you are trying to rebuild causing the well known "dependency hell".

It's recommended that if you want/need to rebuild ports with non-default options, use ports-mgmt/poudriere to create your own package repository. Unfortunately AFAIK, at this time, you are on your own otherwise.

Having said that, there has been some talk about creating package flavours or sub-packages if that's a more appropriate description, though I'm not sure how far along that is or if it's even being worked on at the moment.
 
Interesting. I took some time to read through the portmaster code, and it is indeed disabled if the system uses pkg. Seems like it might not be to hard to add back again.
 
ztp said:
Interesting. I took some time to read through the portmaster code, and it is indeed disabled if the system uses pkg. Seems like it might not be to hard to add back again.
Click to expand...

It is actually hard given that the pkg (and portmgr@) developers have raised the bar on the quality that is acceptable for the packaging utilities. The main problem is that there is no way to ask for a package to be installed via the binary packages that match certain options. For example, if you want a package of security/openssl with SSL3 turned off there's no (and there never was with the old packaging tools either) way to specify that on pkg(8) command line. The ability to use binary packages for fullfilling dependencies was turned off deliberately in order to stop telling lies to the user who blindly trusts that ports-mgmt/portmaster would do the right thing. It never did unless all the options were set at the defaults for all ports involved.
 
Yeah but that would only bring back the unsatisfactory way of using binary packages for dependencies that would always install the package with default options, is that what everyone wants?

The real problem that would need to be solved is package flavors much like you have in OpenBSD. With package flavors the port maintainer could create a set of flavors such as openssl-nossl3 that would place a prebuilt variant of security/openssl in the package repository that has SSL3 turned off and there would be a way to ask for such flavor directly using pkg-install(8).
 
I also wouldn't get my hopes up with the future of ports-mgmt/portmaster. It was very usefull and a needed utility with the old pkg_* packaging utilities because it offered repair functionalities that no other tool did. With the new pkg tools all of those functionalities are now "under the hood" and outside portmaster(8)'s scope leaving it a rather clever wrapper around the basic ports(7) system, not much else.
 
kpa said:
It is actually hard given that the pkg (and portmgr@) developers have raised the bar on the quality that is acceptable for the packaging utilities. The main problem is that there is no way to ask for a package to be installed via the binary packages that match certain options. For example, if you want a package of security/openssl with SSL3 turned off there's no (and there never was with the old packaging tools either) way to specify that on pkg(8) command line. The ability to use binary packages for fullfilling dependencies was turned off deliberately in order to stop telling lies to the user who blindly trusts that ports-mgmt/portmaster would do the right thing. It never did unless all the options were set at the defaults for all ports involved.
Click to expand...

Any program that does package management should have a high standard for code quality, so they are right to be picky. As I posted above, it looks like this is was/is planned to be brought back assuming they merge the pull request I mentioned.

If you want to disable SSL in dependencies of the port you are building then it seems pretty obvious that you won't be able to depend on --packages. The packages are built with whatever options they were built with (whether the default packages or in a custom poudriere repository). If you need this special case then you simply ought not use that option and build everything from source.

That being said, it looks like the information is there to determine which of the dependencies would need to be rebuilt to match build requirements. Commands like pkg search -f <pkgname-x.y.z> and pkg info <pkgname> provide a list of which options they were built with. Sounds rather involved to implement though.
 
kpa said:
Yeah but that would only bring back the unsatisfactory way of using binary packages for dependencies that would always install the package with default options, is that what everyone wants?

The real problem that would need to be solved is package flavors much like you have in OpenBSD. With package flavors the port maintainer could create a set of flavors such as openssl-nossl3 that would place a prebuilt variant of security/openssl in the package repository that has SSL3 turned off and there would be a way to ask for such flavor directly using pkg-install(8).
Click to expand...
Yes, this would be very useful. Much like how in Debian there are -dbg versions of packages that enable debug symbols. But given that this currently doesn't exist, I think the logical default is to install the dependencies with default options. I mean, that's what's built into the packages that you're asking for.
 
I've just encountered the same problem.
There is some solution. It is not very elegant, but it does what I want.

In the ports directory I run
make run-depends-list | sort -n > ~/d-run
and then
make build-depends-list | sort -n > ~/d-build

Then I can diff the output:
diff ~/d-run ~/d-build

First make sure that the required run-depends are installed, then I run pkg add on the packages from the diff output, run make install clean and then run pkg remove on the diff output again. Finally some pkg autoremove to clean up.

If there are some ports in between I want to have special flags, I do the same like described above.
 
Sorry to bump this, but it appears there is an "official" way to do that: in /usr/ports/Mk/bsd.port.mk there is the following section that deals with dependencies:

Code: 
# Show missing dependencies
missing:
        @for dir in $$(${MISSING-DEPENDS-LIST}); do \
                echo $${dir#${PORTSDIR}/}; \
        done

# Show missing dependencies by name
missing-packages:
        @_packages=$$(${PKG_INFO} -aq); \
        for dir in $$(${ALL-DEPENDS-LIST}); do \
                _p=$$(cd $$dir; ${MAKE} -VPKGNAME); \
                if ! $$(${ECHO_CMD} $${_packages} | ${GREP} -q $${_p}); then \
                        ${ECHO_CMD} $${_p}; \
                fi; \
        done

# Install missing dependencies from package
install-missing-packages:
       @_dirs=$$(${MISSING-DEPENDS-LIST}); \
       ${ECHO_CMD} "$${_dirs}" | ${SED} "s%${PORTSDIR}/%%g" | \
               ${SU_CMD} "${XARGS} -o ${PKG_BIN} install -A"
However the `install-missing-packages` verb is buggy, since it passes the port's path to `pkg`. The `missing-packages` however is fine, so replacing with sth along the lines of
Code: 
install-missing-packages:
        @_packages=$$(${PKG_INFO} -aq); \
        for dir in $$(${MISSING-DEPENDS-LIST}); do \
                _p=$$(cd $$dir; ${MAKE} -VPKGNAME); \
                if ! $$(${ECHO_CMD} $${_packages} | ${GREP} -q $${_p}); then \
                        ${ECHO_CMD} "---- Installing dependency: $${_p} ----"; \
                        ${SU_CMD} "${PKG_BIN} install $${_p}"; \
                fi; \
        done
does the trick of passing the correct dependencies to pkg. Now the whole must be called by setting one of the following
Code: 
# USE_PACKAGE_DEPENDS
#                - Try to install dependencies from existing packages instead
#                  of building the port from scratch. Fallback on source
#                  if an existing package is not present.
# USE_PACKAGE_DEPENDS_ONLY
#                - Like USE_PACKAGE_DEPENDS, but do not fallback on source.
either as environment variables or in the configuration file.

Bug filed.
 
Yes it does, but pkg complains since a port has a different name from a package, it would result for example in sth like
pkg install misc/py-torch, which does not work since the package is actually called py311-pytorch.
 
I have no py-torch in my port tree. There is, for example, py-torch-geometric.
If I try pkg install misc/py-torch-geometric, that works and installs among dependencies:
py311-Babel: 2.17.0_1
py311-Jinja2: 3.1.6
py311-aiohappyeyeballs: 2.6.1
py311-aiohttp: 3.11.13
py311-aiosignal: 1.3.2
py311-appdirs: 1.4.4_1
py311-astunparse: 1.6.3_1
py311-attrs: 25.3.0
py311-bottleneck: 1.3.8_1
py311-brotli: 1.1.0
py311-certifi: 2025.1.31
py311-charset-normalizer: 3.4.1_1
py311-contourpy: 1.3.1
py311-cycler: 0.12.1
py311-dgl: 1.1.2_1
py311-dill: 0.3.9
py311-filelock: 3.17.0
py311-fonttools: 4.56.0
py311-frozenlist: 1.5.0
py311-fs2: 2.4.16_1
py311-fsspec: 2025.3.0
py311-gdal: 3.10.2
py311-gmpy2: 2.2.1_1
py311-idna: 3.10
py311-importlib-resources: 6.5.2
py311-joblib: 1.3.2_2
py311-kiwisolver: 1.4.8,1
py311-lxml: 4.9.3_1
py311-markupsafe: 2.1.5_1
py311-matplotlib: 3.8.0_1
py311-mpmath: 1.3.0_2
py311-multidict: 6.2.0
py311-networkx: 3.4,2
py311-numexpr: 2.10.2
py311-numpy: 1.26.4_6,1
py311-olefile: 0.46_1
py311-packaging: 24.2
py311-pandas: 2.1.4,1
py311-pillow: 11.0.0
py311-pluggy: 1.5.0
py311-propcache: 0.3.0
py311-psutil: 7.0.0
py311-pydot: 3.0.4
py311-pygraphviz: 1.6_1
py311-pyparsing: 3.2.1
py311-pysocks: 1.7.1_1
py311-python-dateutil: 2.9.0
py311-pytorch: 2.6.0_3
py311-pytz: 2024.2_1,1
py311-pyyaml: 6.0.1_1
py311-requests: 2.32.3
py311-scikit-learn: 1.4.0_1
py311-scikit-sparse: 0.4.15_3
py311-scipy: 1.11.1_4,1
py311-six: 1.17.0
py311-sqlite3: 3.11.12_10
py311-sympy: 1.13.3_1
py311-threadpoolctl: 3.6.0
py311-tkinter: 3.11.12_10
py311-torch-geometric: 2.6.1
py311-tornado: 6.4
py311-tqdm: 4.67.1
py311-typing-extensions: 4.12.2
py311-tzdata: 2025.2
py311-unicodedata2: 16.0.0
py311-urllib3: 1.26.20,1
py311-wheel: 0.45.1
py311-yarl: 1.18.3
py311-zopfli: 0.2.3
python311: 3.11.12_1
Click to expand...

So, no problem so far. I have never encountered an error with install-missing-packages (except one that comes from a bug in pkg). This is why I'm surprised by your statement.

Do you have a concrete example that generates a not found package?
 
kioftes said:
it would result for example in sth like
pkg install misc/py-torch, which does not work since the package is actually called py311-pytorch.
Click to expand...
pkg(8) accepts a 'pkg-origin', which is the category and name of the port, or in other words, the path of the port. So pkg install misc/py-pytorch is perfectly valid.
 
You must log in or register to reply here.
Back
Top