Ports and dependencies hell

freezr

freezr

Hi Guys,

yesterday day night I had the weird idea to compile LibreOffice with GTK3 support rather than QT. Before to start I checked what dependencies LO would brought down and I copied those in a file as reference.

When I started the process I noticed that more and more ports where required not related with my list of dependencies, I understand that LO is a big and giant piece of software, but quickly everything went out of control, eventually I had to kill the entire process and I rolled back to a previous boot environment without achieving anything.

Looks like that is not really convenient mixing packages with ports since both are unaware about each other but, by any chance, is there any strategies to circumvent the dependencies hell with ports?

Thanks!

tgl
 
tgl said:
Looks like that is not really convenient mixing packages with ports since both are unaware about each other but, by any chance, is there any strategies to circumvent the dependencies hell with ports?
Click to expand...
Looks like you finally woke up to the idea that is actually a Best Practice: Don't mix packages and ports. Pick one or the other when you do a fresh install of FreeBSD and stick with it.
--
I can tell you that even Poudriere suffers from dependency hell. It does try to automate dependency resolution, and there are options to shorten the compilation time by messing with different options in different places. But it's an unwieldy beast that I have been struggling with since June of last year (My latest struggle is making it see updates (https://forums.freebsd.org/threads/poudriere-bulk-not-finding-updates-to-the-tree.83134/), if anyone is curious).
--
Poudriere is also not immune to circular dependencies (packages are, by the virtue of being premade) so you gotta be careful. ?
 
tgl said:
Looks like that is not really convenient mixing packages with ports since both are unaware about each other
Click to expand...

astyle said:
Looks like you finally woke up to the idea that is actually a Best Practice: Don't mix packages and ports. Pick one or the other when you do a fresh install of FreeBSD and stick with it.
Click to expand...
I constantly do that and have no problem. I love both ways and based on the softwares that I want to run, I use them... Also the handbook didn't said anything about the problems of mixing packages with ports.
 
hbsd said:
I constantly do that and have no problem. I love both ways and based on the softwares that I want to run, I use them... Also the handbook didn't said anything about the problems of mixing packages with ports.
Click to expand...
It most certainly does give you a warning about problems of mixing packages and ports:
docs.freebsd.org

Chapter 4. Installing Applications: Packages and Ports

FreeBSD provides two complementary technologies for installing third-party software: the FreeBSD Ports Collection, for installing from source, and packages, for installing from pre-built binaries
docs.freebsd.org docs.freebsd.org
I constantly download ports tarballs from dated commits to HEAD, BTW.
 
Whilst I primarily use packages, I do use ports for a couple of custom builds I need but have noted that sometimes it's not a good idea to build ports on a pkg based system. Sometime, ports and packages are significantly out of sync. Over the last couple of days, there has a been quite a significant disparity between the two as all the plasma based ports (for example) were updated and took a few days for the pkg builds to happen and become available.

There's probably better ways to do it, but I use
portsnap fetch update && portversion -l \< && portversion -l \< | grep -c \
Which updates the ports tree, lists those installed which have an upgrade available and a count of how many there are. This can be a good indication of what will happen if you decide to build a port rather than install/upgrade a pkg.
(Note, each of the slash characters are there to ESCape spaces)
After all, you really don't want to build a quick custom port and find it depends on the latest LLVM, Rust or something equally huge. Wait a day or three for the mirrors to get the latest package update then use something such as portupgrade which will ask for conformation after listing all the ports it's going to upgrade/install. This also gives you the chance to pkg install any build dependencies.
 
rafael_grether said:
make all-depends-list

Will show you all dependencies, including dependencies of LO dependencies …
Click to expand...

?

A dry run of poudriere will take longer, but can show additional detail, for example:

1645212874232.png

1645212664304.png

Less verbosely, non-dry (and cancelled whilst building):

Code: 
root@mowa219-gjp4-8570p-freebsd:~ # poudriere ports -u
[00:00:01] Updating portstree "default" with git+https... done
root@mowa219-gjp4-8570p-freebsd:~ # poudriere bulk -j main -b latest -Ct editors/libreoffice
[00:00:00] Creating the reference jail... done
[00:00:09] Mounting system devices for main-default
[00:00:09] Warning: Using packages from previously failed, or uncommitted, build: /usr/local/poudriere/data/packages/main-default/.building
[00:00:09] Mounting ccache from: /var/cache/ccache
[00:00:09] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:09] Mounting packages from: /usr/local/poudriere/data/packages/main-default
[00:00:09] Mounting distfiles from: /usr/ports/distfiles
[00:00:09] Copying /var/db/ports from: /usr/local/etc/poudriere.d/options
[00:00:09] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/main-default/ref/etc/resolv.conf
[00:00:09] Starting jail main-default
[00:00:09] Will build as nobody:nobody (65534:65534)
[00:00:10] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2022-02-18_19h36m15s
[00:00:10] Loading MOVED for /usr/local/poudriere/data/.m/main-default/ref/usr/ports
[00:00:11] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:11] Inspecting ports tree for modifications to git checkout... yes
[00:00:26] Ports top-level git hash: 495428f817 (dirty)
[00:00:26] Gathering ports metadata
[00:00:31] Calculating ports order and dependencies
[00:00:31] -C specified, cleaning listed packages
[00:00:31] (-C) Flushing package deletions
[00:00:32] Trimming IGNORED and blacklisted ports
[00:00:32] Package fetch: Looking for missing packages to fetch from pkg+http://pkg.FreeBSD.org/${ABI}/latest
Updating FreeBSD repository catalogue...
[main-default] Fetching meta.conf: 100%    163 B   0.2kB/s    00:01   
[main-default] Fetching packagesite.pkg: 100%    6 MiB   3.3MB/s    00:02   
Processing entries: 100%
FreeBSD repository update completed. 31206 packages processed.
All repositories are up to date.
[00:00:45] Package fetch: Will fetch 8 packages from remote or local pkg cache
The following packages will be fetched:

New packages to be FETCHED:
        qt5-widgets: 5.15.2p263 (2 MiB: 100.00% of the 2 MiB to download)

Number of packages to be fetched: 1

The process will require 2 MiB more space.
2 MiB to be downloaded.
[main-default] Fetching qt5-widgets-5.15.2p263.pkg: 100%    2 MiB   2.5MB/s    00:01   
[00:01:29] Package fetch: Using cached copy of qt5-widgets-5.15.2p263
[00:01:29] Package fetch: Using cached copy of qt5-x11extras-5.15.2p1
[00:01:29] Package fetch: Using cached copy of doxygen-1.9.3,2
[00:01:29] Package fetch: Using cached copy of tex-dvipsk-5.995_2
[00:01:29] Package fetch: Using cached copy of tex-formats-20150521_2
[00:01:29] Package fetch: Using cached copy of texlive-base-20150521_77
[00:01:29] Package fetch: Using cached copy of tex-basic-engines-20150521
[00:01:29] Package fetch: Using cached copy of texlive-texmf-20150523_4
[00:01:29] Sanity checking the repository
[00:01:29] Checking packages for incremental rebuild needs
[00:01:33] Deleting doxygen-1.9.3,2.pkg: missing dependency: graphviz-2.44.1_18
[00:01:34] Deleting qt5-widgets-5.15.2p263.pkg: missing dependency: qt5-gui-5.15.2p263
[00:01:34] Deleting qt5-x11extras-5.15.2p1.pkg: missing dependency: qt5-gui-5.15.2p263
[00:01:34] Deleting tex-basic-engines-20150521.pkg: missing dependency: tex-web2c-20150521_3
[00:01:34] Deleting tex-formats-20150521_2.pkg: missing dependency: tex-basic-engines-20150521
[00:01:34] Deleting tex-dvipsk-5.995_2.pkg: missing dependency: tex-web2c-20150521_3
[00:01:34] Deleting texlive-base-20150521_77.pkg: missing dependency: tex-web2c-20150521_3
[00:01:34] Deleting texlive-texmf-20150523_4.pkg: missing dependency: texlive-base-20150521_77
[00:01:34] Deleting stale symlinks... done
[00:01:36] Deleting empty directories... done
[00:01:36] Package fetch: Generating logs for fetched packages
[00:01:36] Unqueueing existing packages
[00:01:36] Unqueueing orphaned build dependencies
[00:01:36] Sanity checking build queue
[00:01:36] Processing PRIORITY_BOOST
[00:01:36] Balancing pool
[main-default] [2022-02-18_19h36m15s] [balancing_pool:] Queued: 13 Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 13  Time: 00:01:26
[00:01:36] Recording filesystem state for prepkg... done
[00:01:36] Building 13 packages using up to 2 builders
[00:01:36] Hit CTRL+t at any time to see build progress and stats
[00:01:36] [01] [00:00:00] Builder starting
[00:01:36] [02] [00:00:00] Builder starting
[00:01:46] [02] [00:00:10] Builder started
[00:01:46] [02] [00:00:00] Building devel/mdds | mdds-2.0.1
[00:01:46] [01] [00:00:10] Builder started
[00:01:46] [01] [00:00:00] Building graphics/mesa-libs | mesa-libs-21.3.6
[00:01:55] [02] [00:00:09] Finished devel/mdds | mdds-2.0.1: Success
[00:01:55] [02] [00:00:00] Building textproc/libixion | libixion-0.17.0
load: 2.52  cmd: sh 57545 [piperd] 110.06r 0.00u 0.00s 0% 3492k
mi_switch+0xc2 sleepq_catch_signals+0x2e6 sleepq_wait_sig+0x9 _sleep+0x1e3 pipe_read+0x3d6 dofileread+0x81 sys_read+0xbc amd64_syscall+0x10c fast_syscall_common+0xf8
[main-default] [2022-02-18_19h36m15s] [parallel_build:] Queued: 13 Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 12  Time: 00:03:16
 ID  TOTAL                ORIGIN   PKGNAME              PHASE PHASE    TMPFS     CPU% MEM%
[01] 00:01:40 graphics/mesa-libs | mesa-libs-21.3.6     build 00:00:18 2.65 GiB 84.2% 1.5%
[02] 00:01:31  textproc/libixion | libixion-0.17.0  configure 00:01:16 1.08 GiB   22% 1.1%
[00:03:26] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2022-02-18_19h36m15s
load: 2.38  cmd: sh 57545 [piperd] 144.58r 0.00u 0.00s 0% 3488k
mi_switch+0xc2 sleepq_catch_signals+0x2e6 sleepq_wait_sig+0x9 _sleep+0x1e3 pipe_read+0x3d6 dofileread+0x81 sys_read+0xbc amd64_syscall+0x10c fast_syscall_common+0xf8
[main-default] [2022-02-18_19h36m15s] [parallel_build:] Queued: 13 Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 12  Time: 00:03:51
 ID  TOTAL                ORIGIN   PKGNAME          PHASE PHASE    TMPFS    CPU% MEM%
[01] 00:02:15 graphics/mesa-libs | mesa-libs-21.3.6 build 00:00:53 2.65 GiB   0% 0.1%
[02] 00:02:06  textproc/libixion | libixion-0.17.0  build 00:00:23 1.08 GiB   0% 0.8%
[00:04:01] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2022-02-18_19h36m15s
^C[00:04:04] Error: Signal SIGINT caught, cleaning up and exiting
[main-default] [2022-02-18_19h36m15s] [sigint:] Queued: 13 Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 12  Time: 00:03:54
[00:04:04] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2022-02-18_19h36m15s
[00:04:04] Cleaning up
main-default: removed
main-default-n: removed
[00:04:04] Unmounting file systems
root@mowa219-gjp4-8570p-freebsd:~ #
 
From that warning:
If the Ports Collection and pkg must be used in conjunction, then be sure that your Ports Collection and pkg are on the same branch release of the ports tree.
Click to expand...
It does take some effort to set up and verify that. But if you're gonna mix, you kind of gotta put in that effort, or else divergent dependencies will bite at a bad time.
 
That's an opinion that got inserted into the handbook. An opinion shouldn't be set in stone as a truth. There are many users who don't do that, and that should be left to preference. I doubt that used to be there. The handbook took a step back, and it was better then, with using the forums to fill in obsolete information.
 
sidetone said:
There are many users who don't do that
Click to expand...
Because it is a lot of work, and most people don't have the patience to do that. A solid understanding of directed acyclic graph theory helps explain the reasoning behind that warning. Those who have not been bitten by the divergent dependencies yet - they haven't done enough upgrading to get to that point. Can you connect the dots between DAGs and dependency hell? ;) Cal-Berkeley University most certainly teaches undergrad classes on these subjects.
 
freezr said:
Looks like that is not really convenient mixing packages with ports since both are unaware about each other but, by any chance, is there any strategies to circumvent the dependencies hell with ports?
Click to expand...
ports-mgmt/portmaster will pull in a list of dependencies it wants to install with a program and give you the chance to select options before moving to the next. Unless it's an option you have a reason to change, like to include nmap with the rkhunter build, I change very few options and advise you not to.

From that point on, unless it asks you if you want to keep or delete an old file, it should work till it's done. I always exit the WM to the login screen to work with ports, sit in my recliner and can tell at a glance if it is done, stopped for instructions, has balked and requires needs humanoid intervention or do it myself if none are available.

I've been using ports since 2005 and taught myself to use them through trial and error so there is very little I haven't seen it do or problem I haven't worked though. Sometime mixing pkg with ports is the best way I see to move past a point and won't hesitate to mix them if that will get me going.

I can't really say it's caused me a problem. I can say it's fixed a problem many times and don't sweat it, but that's me. I never ask questions about anything. Not someone warning you not to do something because they can't do it themselves, lay claim to a lot of lofty knowledge they don't posses and ask the most questions.

A grain of salt sometime required. To cleanse the palate, if youi puke in your mouth a little.
 
You must log in or register to reply here.
Back
Top