pkg repositories provided by the FreeBSD project needs to grow up (sigh)

[zirias@]

In 25 years using different Linux distributions, I've never seen such behaviors.
I've been using Void - a rolling release distribution - for 2+ years now and it never happened either.

So, if you tell me that there are rolling release distributions that never experience package build errors, I simply don't buy it.
If you just tell me they handle it differently (by just keeping the latest successful build), that's a different story, and it's kind of "hiding" the problem. Whether that's better is probably open for debate -- it sure avoids the typical temporary unavailability of packages, but the cost could be subtle incompatibilities with "funny" results at runtime.

 

[zirias@]

in Debian/Devuan the upgrades are pushed in to the repository when all the dependency chain as a hole is resolved.

There are a lot of independent package updates in Debian sid. There are coordinated port upgrades (for stuff like Qt, KDE, ...) in FreeBSD ports head. Not sure where you see the conceptual difference...

 

[Mjölnir]

To go back to the genuine topic: One can pkg help lock vital packages and every 3 month or so update a jail or VM to see if such problem comes up. Then wait to upgrade the main workstation/server.

 

[tingo]

To go back to the genuine topic: One can pkg help lock vital packages and every 3 month or so update a jail or VM to see if such problem comes up. Then wait to upgrade the main workstation/server.

From a user point of view that is a workaround (band-aid) for a system which is not working.

 

[jmos]

I have not much experince on Linux as a hole , but in Debian/Devuan the upgrades are pushed in to the repository when all the dependency chain as a hole is resolved. So there is no problem and also dist-upgrade for remaining unresolved issue.

Debian knows stable, testing and sid ("unstable" / "still in development"). On sid or testing you can get unmet dependencies. In testing this happens seldom to the commonly used packages, but it happens. And stable has a freeze on packages, so there are months to solve those problems. And that's the price of it: Older software.

 

[Mjölnir]

So it turn out to be a matter of policy. The mantainers do the pkg lock for you. That's not the philosophy of FreeBSD's handling of the ports collection. If you want that, maybe one of the FreeBSD-based distributions like Ghost/Fury/MidnightBSD better fits your needs. PC_BSD/TrueOS had sandboxed application bundles (different library versions can co-exist), you might be able to find the sources for that framework or maybe it's in the ports?

 

[zirias@]

As already said, the quick solution to avoid these problems is using quarterly. It's a sane default for using packages. It's not nearly as old as Debian stable, and most of the time, there's no attempt to "backport" upstream fixes, but if necessary, just import the new version. Plus when the branch is still new, build problems are more likely, so maybe just don't upgrade the first few days of every quarter. But apart from that, it should be at least as "stable" as Debian testing. Definitely a reasonable choice.

 

[gnath]

I think problem is non-availability of some packages from repository. It is not in debian (all 3 branches). Devuan took almost one year to unchain tangled dependency due to one systemd. Even security update took some time.
qt5-network depends on three and required by three ports. OpenSSL is now missing. Now you settle the situation keeping OpenSSL and think similar situation as whole.

As noted above by all others, it's a matter of the build process. When 150 of 30,000 ports fail to build, that's 0.5%

On the last build run 14 ports failed to build and 1318 ports were skipped (probably because a dependency failed).

What do you suggest as a possible solution?

No quick solution except clearing the tangle. What I understand that quarterly and latest at that day are build from same revision of head 'branch'.

 

[20-100-2fe]

Ok, so lets turn this around. What do you suggest should happen when a port fails to build and thus a package cannot be created? The port fails and there's no simple way to fix it for a particular legacy release version. The port works fine on newer releases. What do you suggest as a possible solution?

The possible solution I suggest is to use the same principles as those adopted everywhere else.
I don't mean just Linux package managers, but also maven, npm, composer...
There is a reason for all of them being built on the same principles.
Maybe this reason is that it's the best strategy we've found so far to handle dependencies and repository integrity, don't you think?
Or are all of them wrong?
Why should we deprive FreeBSD of the benefits of the learnings and findings of other developers?

 

[zirias@]

20-100-2fe You don't explain what *you* think the key difference would be (and how this is "better")? I don't see any deficiencies with dependency handling in pkg. The occassional problems people run into are ~90% using latest and some packages are temporarily missing for build failures. So, asking again, is your suggestion to just keep the latest successful build of each individual packages? As also mentioned before, this has other drawbacks. If you see any drawbacks regarding dependencies, care to explain?

 

[20-100-2fe]

20-100-2fe You don't explain what *you* think the key difference would be (and how this is "better")? I don't see any deficiencies with dependency handling in pkg. The occassional problems people run into are ~90% using latest and some packages are temporarily missing for build failures. So, asking again, is your suggestion to just keep the latest successful build of each individual packages? As also mentioned before, this has other drawbacks. If you see any drawbacks regarding dependencies, care to explain?

The other package management systems keep all versions of the packages, so when a build fails (or is not finished) you don't have the same problems as with FreeBSD, i.e. not being able to install the package, or having pkg upgrade delete your installed package because it is no longer in the repository.

With the other systems, a package is only upgraded when all its dependencies are available in the repository.
As long as they aren't, you keep - or are able to install - the previous version.

Doing this for FreeBSD would probably not require much work : adding tests in pkg and keeping previous versions of the packages in the repository.
And it would also trivially solve the issues we have with drm-kmod at each release.

 

[Mjölnir]

The drawback would be that a single or few ports can delay the whole progress by several month, s/t even a year. In the meantime, the maintainers have to provide at least security fixes for the versions in the repository, while also working to fix the new version's dependencies. Who's going to do that additional work?

 

[zirias@]

The other package management systems keep all versions of the packages, so when a build fails (or is not finished) you don't have the same problems as with FreeBSD, i.e. not being able to install the package, or having pkg upgrade delete your installed package because it is no longer in the repository.

That's possible, but has other drawbacks -- it can lead to "funny" bugs at runtime. Still, you could argue about what's worse for the end user.

With the other systems, a package is only upgraded when all its dependencies are available in the repository.

"The other systems" also have some form of package builds, and for this "source", having all the dependencies upfront is the only way this could ever work -- no different with FreeBSD ports. Of course, in a binary repository, you should also only put packages when all their dependencies are available as well -- which is btw exactly what poudriere is doing. I get the feeling you are confusing something here, it's all coming back to the question how you deal with unexpected build fails. Poudriere just doesn't add that package to the repository, and as a consequence, all dependent packages aren't added either. The only alternative would be to keep the latest successful build, but see above ...

Build fails happen, and of course, they happen for e.g. Debian sid as well. Probably, Debian takes the approach of just keeping the previously built packages in their binary repository. But that's trading the problem of temporarily unavailable packages for the problem of unexpected runtime instability.

If you want a more stable set of packages, FreeBSD offers that in the quarterly repository built from the current quarterly branch.