Proposal: a pkg upgrade mode that never silently drops installed packages

ZioMario

ZioMario

Hello to everyone.

I realized that when I do a "pkg upgrade",some installed packages silent drops : it means that they are removed without notice. For this reason,I asked Claude to elaborate a strategy for making a better upgrading / installing packages strategy.

Since I do "vibe coding",no one will be interested to the code I produce,so I will not be involved personally in this project. Anyway,I want to propose the plan to someone that can code it personally,using his brain :

So,this is what Claude (Opus 5.5) suggested :

1. It's not exactly "silent", but the policy is wrong.

pkg upgrade does show an Installed packages to be REMOVED section and asks for confirmation, unless -y or ASSUME_ALWAYS_YES is set. In a 200-line plan, though, that section is easy to miss. The real flaw is elsewhere: pkg's SAT solver (picosat) treats removal as an operation just as legitimate as an upgrade. There is no --never-remove-style option that turns a removal into an error.

2. The ecosystem limit.

FreeBSD repositories (quarterly/latest) carry only one version per origin. If py39-foo disappears from the repo or conflicts with the new py311 default, no solver can "find the best combination" using the repo alone, because that combination doesn't exist. The only way to give the solver a choice is to make additional versions available to it. So this is not just a better solver: it also needs a richer catalog.

Design​


A. Snapshot of the initial state.

For every installed package, record: origin, flavor, version, automatic flag, lock state, shlibs_provided and shlibs_required (via pkg query). Before touching anything, create a ZFS boot environment with bectl create.

B. Multi-source catalog with priorities.
  1. Remote repo (quarterly or latest).
  2. A local repo generated with pkg repo over /var/cache/pkg. It contains the currently installed packages, so "keep the old version" is always an available choice.
  3. Optional: a personal poudriere repo, for ports removed from the official tree.

C. Problem model (MaxSAT / pseudo-boolean, CUDF/Mancoosi style).

Hard constraints:
  • dependencies (origin/flavor);
  • declared conflicts and file conflicts;
  • at most one version per origin;
  • every shlib_required of the selected packages must be provided by a selected package or by the base system.
That last point is decisive. pkg metadata already includes shlibs, so the solver can verify whether an old package kept from the cache still works with the new libraries, instead of keeping it blindly and ending up with a broken binary.

D. Lexicographic objective, in strict order: a lower level can never compensate for a worse result at a higher one.
  1. Zero removals of explicitly installed (non-automatic) packages.
  2. Minimum number of removals of dependencies that are still required.
  3. Minimum number of packages kept at an old version or downgraded.
  4. Maximum number of packages brought to the newest version.
  5. Minimum number of newly installed packages.
E. If level 1 cannot reach zero, the system does not proceed.

It extracts the minimal unsatisfiable subset (MUS) and reports it in readable form, for example: "X cannot stay because it requires libfoo.so.3, provided only by Y, which conflicts with Z, required by W". It then lists the concrete alternatives: keep W at the old version, remove X, or build X in poudriere. The decision stays with the user and is never made implicitly.

F. Execution.

The plan is translated into explicit operations on specific .pkg files, not into a generic pkg upgrade that would recompute everything from scratch. Packages kept at an old version get pkg lock, so the next bare pkg upgrade doesn't throw them away.

G. Final verification.

Diff the initial snapshot against the final one and check shlibs with pkg check -d and pkg check -B. If something doesn't add up, roll back to the boot environment.

Implementation​

  • libsolv (BSD-3 license, the solver used by zypper and dnf). It already has the concepts of installed packages to preserve, "lock" and "favor" jobs, and problem explanations with alternative solutions. The main work is converting the pkg catalog (packagesite.yaml, one JSON object per line, with shlibs as "provides") into libsolv's format.

  • Alternative: direct MaxSAT with PySAT/RC2 (MIT). More control over the lexicographic objective, but MUS extraction and the explanations have to be written from scratch.
libsolv is the lower-risk choice: it is mature and has been in production for years on exactly this problem.

Immediate mitigation with current pkg​

A wrapper that runs pkg upgrade -n and aborts if the to be REMOVED section is not empty. It's crude and doesn't find alternatives, but it removes the surprise right away. There is also the BACKUP_LIBRARIES option in pkg.conf, which keeps removed shared libraries in a compatibility package and reduces ABI breakage for packages kept at old versions. It needs to be verified against the installed pkg version.
 
Why not get Claude to read the man pages

just use the -n option for a dry run
and create a bectl boot environment before you upgrade

pkg-upgrade - FreeBSD 15.1-RELEASE and Ports.quarterly Manual Pages

man.freebsd.org man.freebsd.org

Code: 
     -n, --dry-run
         Dry-run mode: show what packages have updates available, but do
         not perform any upgrades.  Repository catalogues  will  be  up-
         dated as usual unless the -U option is also given.

Code: 
doas pkg upgrade -n

Code: 
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
    glib-bootstrap: 2.84.4,2 -> 2.86.4,2 [FreeBSD-ports]

Number of packages to be upgraded: 1

there is already a pkg cache of installed packages on your machine
if memory serves me right

yes my memory was correct

Code: 
ls -l /var/cache/pkg
 
You must log in or register to reply here.
Back
Top