Dependency bloat in the ports collection

A lot of things require Rust. A lot of programs will require Rust anyway. I still don't get why you keep saying you need Rust for Xorg or its drivers. Have things changed that much radically most recently for that to be needed for it? Whenever I rebuilt my Xorg recently, things break.

As for LLVM, different sets of packages insist on two different versions of it. Actually, all that's really needed is not the full compiler, just the build utilities equivalent to binutils. GNU Binutils will actually do (or has recently). They put utilities and compilers into a monolithic package for each version.

While it's often discouraged to tinker with /usr/ports/Mk/ and /usr/ports/Mk/Uses/, versions for all programs can be set to one needed ports LLVM, then give feedback to maintainers or the mailing list on if that works. Then, be ready to overwrite the ports/ directory with a fresh svnup or svn. The one that won't likely build is the Xorg drivers that actually do require the latest version of LLVM (actually its utilities). So, it should only need the base LLVM plus the latest one from ports (but not necessarily the latest build). Furthermore, it's likely only the latest utilities for LLVM that are needed from the latest version, which aren't separated from the LLVM compiler.

Maybe the solution is to ask for them to separate LLVM's utilities from the LLVM compiler build. So that dependencies don't ask for hours of additional compile time, when something actually needs the latest compiler utilities, and not the whole LLVM set. Also, so that the compiler in the base system is enough for everything requiring LLVM/Clang.

GNU Binutils at a recent time had the needed components to build everything on various processors. LLVM's Clang utilities are limited, many parts of it only work for a limited amount of processors, and parts of it are not finished. Parts of LLVM Clang build for everything across all processors. When a small improvement is made, they set it to download a newer update of LLVM/Clang by default.

LLVM/Clang itself isn't the problem. The problem is the utils which are unfinished. I believe parts from GNU binutils are used to fill in the gaps. Not all programs need all of the build utilities or the latest ones, but some that ask for the latest LLVM version do or have in a most recent time.


The quickest fix is to see which versions of LLVM and Rust are wanted, cancel that build, then install these two (plus a Rust dependency) from packages. Then use the -i option with portmaster for your desired program to avoid rebuilding LLVM and Rust, when a package for the latest build for these is unavailable. Using packages from the lang category for the most part doesn't interfere with building from ports.
 
Diverting just a little, recently some new ports which require rust want to rebuild it even though it is already installed, meaning "wait and see if there is a package"
 
Right, two days and a bit of down-winding later. First, I appologize for the tone of my first post. Never post when you're po'ed about something.

I understand that most people install from binary packages, which of course is the sensible option when you configure a machine that you actually want to use, but that's not the purpose of the machine I was ranting about. I should probably have made that clearer.

I'm doing the build testing for CDE, the legacy Unix desktop that many old hacks like myself started their career on and still can't let go of. This being legacy software, you are of course vulnerable to suffer breakages from newer packages. Case in point: GCC 10, which made -f-no-common a default flag. That in turn wrecked all our builds on every distro that updated to gcc10. (Mainly the Redhat and Debian derivates), so I need to check the latest upstream builds.

And this is where my disappointment with the current state of OSS comes in. The FreeBSD ports are only the most extreme example, other package management systems have the same problems to a degree - feature creep and dependency bloat.

Take VIM for example. That thing has GUI options ranging from Xaw to GTK3 - guess which one is the default? Right, GTK3, so, unless you wade through one hundred eleventy config menus or write an mk.conf on NetBSD or make.conf on FreeBSD, which then isn't always honoured by the look of things, you have a hard time automating stuff, and if you go by default, you build half the OSS world that you then don't need. I guess that's why people moved away from building from source in the first place.

So my wish here would be to go with sensible minimal defaults. If people want all the bells and whistles, let them add the features, not requiring me to disable them. Every added feature usually introduce whole smorgasboard of new dependencies. Why not go for the minimum? In the case of VIM that would be no GUI at all. Please let me decide that I want one, and if so, which one.

As for librsvg - I really need to bite my lip on that one. Sure, the authors of it have every right in the world to ditch a language that has worked perfectly well for decades (and still does) in favour of going for the latest fad - all power to them, but it exposes a problem that has been around in the opensource world for quite a time now: people don't care about other people's work as much as they should.

The success of an OSS project (especially a library) is measured by how many people choose to use it for their project. In that regard, I'd say librsvg doesn't do too badly. But that success comes with consequences. If you make a major decision for your project, it affects all those who chose to rely on it. So by going rust, you force other people to abandon their reliance on your software or deal with stuff they never saw coming and were never asked about. That, and a 2 hour time penalty when building the now new dependencies for your own project. Not to mention that you suddenly depend on code written in a language you are not overly familiar with. And if I learned something in the 80s - don't mix languages. Everybody who ever tried to import a Turbo-Pascal library into a Turbo-C project will know what I mean.

What would have been the damage in just sort of sealing librsvg as a sort of 'finished project', so people who are satisfied with it can stick with it, and rustifying it under a new name? librsvg-ng or something.

Sorry, I was rambling again, but these are my thoughts on it.
 
Agreed. I think your "rambles" are pretty much on the mark too. I am also fairly annoyed by the sheer number of dependencies that software these days bring in. Part of this is the nature of "modern" software being a sprawling mess and the other part of this is the FreeBSD ports system being a little too inclusive of features that only one or two people care about.

The issue is that I am not sure there is a solution that will please everyone. I personally would like a stripped ports collection where the absolute minimum of features (and thus dependencies) is the default. However this would likely upset a few people. For example would Vim support the X11 clipboard? Would Mutt support imap?

I was on the original team that open-sourced CDE (one of my patches also has a visual bug on the dtpanel that I plan to fix too!). However I kind of pulled away once clang came in vs gcc and I was far too over my head in terms of time needed to patch it to compile. I have much respect for those who achieved this!

In fact I am recently developing a UI toolkit with zero dependencies other than libX11 to try to address this very problem (albeit in a minor way for now).

You might consider using packages but then use ports only for "leaf" software. So any software that is not depended on by others is generally safe to be mixed with packages. I tend to do this. A recursive make on any port is normally... not a good use of the family computer time XD.
 
As for librsvg - I really need to bite my lip on that one. Sure, the authors of it have every right in the world to ditch a language that has worked perfectly well for decades (and still does) in favour of going for the latest fad - all power to them, but it exposes a problem that has been around in the opensource world for quite a time now: people don't care about other people's work as much as they should.
The commit that added the non-rust version to default versions mentions that there are platforms on which Rust is not available. I'm hoping this leads to a fork. Hopefully there are enough people that care about resource-constrained environments where Rust doesn't make sense. I'm not against Rust on principle. It's clearly not ready for prime time, though.
 
  • Thanks
Reactions: a6h
The commit that added the non-rust version to default versions mentions that there are platforms on which Rust is not available. I'm hoping this leads to a fork. Hopefully there are enough people that care about resource-constrained environments where Rust doesn't make sense. I'm not against Rust on principle. It's clearly not ready for prime time, though.
Why does rust doesn't make sense on a resource constrained environment? llvm11 or gcc10 are better in this regard? Do you know how many gigabytes of disk space / ram llvm11 requires to build?
 
Why does rust doesn't make sense on a resource constrained environment? llvm11 or gcc10 are better in this regard? Do you know how many gigabytes of disk space / ram llvm11 requires to build?
It's not just that you have to have Rust in addition to at least one of those two, it's also that compiling with Rust takes far longer and is much more resource intensive than compiling C.
 
In case people have been following along and are wondering what happened. Apparently our tolerance was mistaken for an inability to act. As a result getopt received a couple of warning points and a temporary ban. The disruptive posts have been removed.
 
I don't think that has anything to do with Rust in particular. I've been having build problems due to resource starvation for a while too, not all of them related to Rust. I had a period when building actually caused the build server to crash and reboot. Quite annoying. Seems to have settled down a bit now, at least for my builds, some things still take forever but at least its not crashing halfway through a poudriere run any more.
 
I am building quarterly ports with ports-mgmt/poudriere on my desktop machine while using it for other purpose. I only have 8Gb of memory and use 3 cores of my AMD FX6300. In /usr/local/etc/poudriere.conf I have the setting USE_TMPFS=no and don't go out of swap space (8G). My filesystem is ZFS but I am limiting ARC with vfs.zfs.arc_max="3G" in /boot/loader.conf.
 
I have to say that building www/firefox-esr launches rust instances on all my cores which is greatly slowing down my computer despite starting poudriere with nice +20. But after a while all is returning to normal situation.
 
Maybe only an upgrade to Clang or build utils is needed. LLVM and Clang are in the same port or package. LLVM shouldn't have to be recompiled to get the recent version of Clang or the build utilities (Bin/Elf-utils) in the case where only a port demands it, rather than the user wants to use the whole set of LLVM/Clang.

Is it correct that Rust uses LLVM that's in base or the default one in ports? I know it's said that Rust uses LLVM. I'm trying to see if this is how it's implemented from using the one in base or the default one from make.conf.
 
Back
Top