Dependency bloat in the ports collection

dpecher

New Member

Reaction score: 9
Messages: 6

Hello everybody,

I'm sorry that my first post in here has to be a critical one, but I'm seriously nearing the end of my tether. We've come to a point where just compiling the bare-bones xorg port takes 12 hours and installs two different llvm versions and that infernal rust junk that compiles in geological terms and seems to be neccessary only for FreeBSD. I've never had to install that malarkey on any other system, ever.

Normally I'm not even a regular FreeBSD user anymore - you've lost me when CLang was chosen to supplant a perfectly servicable compiler, but since I'm doing a lot of platform testing as a member of the CDE project team, I still have the latest non-eol'ed FreeBSDs running on my ESX rack. Theoretically I could install from binaries of course, but go back to FreeBSD 10 and some of the packages were compiled by Moses - not very helpful when testing a new software version of your own stuff.

So here the question: What is it with that dependency bloat? Why do packages compile dependencies that they definitely do not need, and more importantly, is there any action planned to remedy this. At the moment we cannot in good conscience ask our users to compile the latest CDE version if building xorg and git ruins two perfectly usable days of your life.

Sorry for the rant,
Danilo Pecher
 

shkhln

Daemon

Reaction score: 749
Messages: 1,826

you've lost me when CLang was chosen to supplant a perfectly servicable compiler,
You lost whatever argument you had right there. The only thing you can expect from this point is more flame.

but go back to FreeBSD 10
Please, don't go back to FreeBSD 10, it's EOL.

So here the question: What is it with that dependency bloat? Why do packages compile dependencies that they definitely do not need, and more importantly, is there any action planned to remedy this.
Nothing will be done about that. Defaults are intentionally not minimal to cover as many use cases as possible.

At the moment we cannot in good conscience ask our users to compile the latest CDE version if building xorg and git ruins two perfectly usable days of your life.
Who cares? (Xorg doesn't have dependency on Rust, by the way.)
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,783
Messages: 34,910

We've come to a point where just compiling the bare-bones xorg port takes 12 hours
What are you using, a 486? Even my old Core i5 does this in less than an hour.

and installs two different llvm versions
Nope.
Code:
root@molly:/usr/ports/x11/xorg # make all-depends-list | grep llvm
/usr/ports/devel/llvm10

and that infernal rust junk that compiles in geological terms and seems to be neccessary only for FreeBSD.
Rust as a dependency of Xorg? Whatever gave you that idea? I think you got things quite backwards here.
I've never had to install that malarkey on any other system, ever.
Try building Firefox from source without it. On any other system.

Theoretically I could install from binaries of course, but go back to FreeBSD 10 and some of the packages were compiled by Moses
FreeBSD 10 has been EoL for a couple of years, what where you expecting?

not very helpful when testing a new software version of your own stuff.
Why are you testing on an EoL version in the first place?
 

getopt

Aspiring Daemon

Reaction score: 613
Messages: 866

Folks, don't play this down or smile it away. While he could have written a little more conciliatory he has a point!

I have seen building three different versions of LLVM in Poudriere at a time. And the last Rust stupidity was using it as a standard dependency for LIBRSVG2.

But! Mostly the problems come from "upstream". What could be done fom ports maintainers is to make wise use of port options and prefer those form a minimalistic approach.

Also I'd like to use a tool for reviewing the dependency-tree before setting port options. A lot can and should be done to make the dependency jungle more transparent for those building their packages from source.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,783
Messages: 34,910

And the last Rust stupidity was using it as a standard dependency for LIBRSVG2.
There are two different ports for it; graphics/librsvg2 and graphics/librsvg2-rust. These two conflict. Gnome for example seems to depend on the Rust version. I suspect Xorg may have changed accordingly because of this conflict. Or else you would never be able to install Gnome and Xorg packages together. Imagine the flak we would get from that.
 

getopt

Aspiring Daemon

Reaction score: 613
Messages: 866

For time being you can use
Code:
DEFAULT_VERSIONS+=librsvg2=legacy

to prevent the Rust port. But this won't hold for eternity.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,783
Messages: 34,910

Updated my ports tree as I thought I may have been looking at old info but Xorg doesn't appear to depend on librsvg at all. So I'm still wondering why Rust would be a dependency of Xorg.
 

getopt

Aspiring Daemon

Reaction score: 613
Messages: 866

I cannot see the OP relating Rust to Xorg. From my reading he addresses this in addition to the multiple LLVM-versions problem in the ports.

Having a base-LLVM should just be enough, IMO.
 

Argentum

Active Member

Reaction score: 57
Messages: 174

I'm sorry that my first post in here has to be a critical one, but I'm seriously nearing the end of my tether. We've come to a point where just compiling the bare-bones xorg port takes 12 hours and installs two different llvm versions and that infernal rust junk that compiles in geological terms and seems to be neccessary only for FreeBSD. I've never had to install that malarkey on any other system, ever.
Hard to tell, what exactly are you doing, but just out of curiosity just tried to recompile my X-org. It took less than 2 minutes.
 

Argentum

Active Member

Reaction score: 57
Messages: 174

Updated my ports tree as I thought I may have been looking at old info but Xorg doesn't appear to depend on librsvg at all. So I'm still wondering why Rust would be a dependency of Xorg.
True.

pkg_tree -v xorg-server-1.20.9,1|grep librsvg gives nothing.

pkg_tree xorg-server-1.20.9,1

Code:
xorg-server-1.20.9,1
|\__ xkeyboard-config-2.31
|\__ xkbcomp-1.4.3
|\__ pixman-0.40.0_1
|\__ libxshmfence-1.3
|\__ libxkbfile-1.1.0
|\__ libXdmcp-1.1.3
|\__ libXau-1.0.9
|\__ libXfont2-2.0.4
|\__ openssl-1.1.1h_1,1
|\__ mesa-libs-20.2.0_1
|\__ mesa-dri-20.2.0_1
|\__ libepoxy-1.5.4
|\__ libdrm-2.4.103,1
|\__ libunwind-20200331_1
|\__ libudev-devd-0.4.2_1
 \__ libpciaccess-0.16
 

ShelLuser

Son of Beastie

Reaction score: 1,871
Messages: 3,636

While the OP seems to be a bit of a drama queen he still has a fair point when it comes to LLVM:

Code:
peter@vps:/home/peter $ pkg info -x llvm
llvm10-10.0.1_3
llvm11-11.0.0
llvm90-9.0.1_3

This is the situation on my VPS server and I don't have anything out of the ordinary running here, merely Apache backed up by Mono, Java (through Tomcat) and PHP using PostgreSQL and MySQL as backend database systems. Mail is handled by Postfix. To add insult to injury I even use DEFAULT_VERSIONS in /etc/make.conf where I have llvm set to 90, as suggested by /usr/ports/Mk/bsd.default-versions.mk.

Yet here we are. Of course now that I have these versions installed I won't be confronted with another build any time soon, but it's still pretty ridiculous to have 3 versions of the same thing installed, especially considering that the version within base could have handled things as well.

There are moments when it truly seems as if FreeBSD is more busy building compilers than the actual software we want to use. It's annoying for sure, but not something to start a whole drama over I think. But I do agree that some quality guidelines might be a good thing here.

At the very least have ports honor /etc/make.conf.
 

Argentum

Active Member

Reaction score: 57
Messages: 174

While the OP seems to be a bit of a drama queen he still has a fair point when it comes to LLVM:

Code:
peter@vps:/home/peter $ pkg info -x llvm
llvm10-10.0.1_3
llvm11-11.0.0
llvm90-9.0.1_3

This is the situation on my VPS server and I don't have anything out of the ordinary running here, merely Apache backed up by Mono, Java (through Tomcat) and PHP using PostgreSQL and MySQL as backend database systems. Mail is handled by Postfix. To add insult to injury I even use DEFAULT_VERSIONS in /etc/make.conf where I have llvm set to 90, as suggested by /usr/ports/Mk/bsd.default-versions.mk.

Yet here we are. Of course now that I have these versions installed I won't be confronted with another build any time soon, but it's still pretty ridiculous to have 3 versions of the same thing installed, especially considering that the version within base could have handled things as well.
Agree! The only question is, who should deal with this problem?

BTW, in my laptop - pkg info -x llvm

Code:
llvm10-10.0.1_3
llvm11-11.0.0
llvm80-8.0.1_4
llvm90-9.0.1_3


And looking portversion -vr llvm10-10.0.1_3

has the only llvm which has dependency:
Code:
[Reading data from pkg(8) ... - 1460 packages found - done]
llvm10-10.0.1_3             =  up-to-date with port
mesa-dri-20.2.0_1           =  up-to-date with port
 

tingo

Daemon

Reaction score: 532
Messages: 2,369

But! Mostly the problems come from "upstream". What could be done fom ports maintainers is to make wise use of port options and prefer those form a minimalistic approach.
True. But they didn't do that. As a post on the freebsd-ports mailing list recently told, it seems like the policy / accepted practice for ports / port maintainers is to follow upstream as close as possible.

TL;DR - yes there is a "dependency bloat" in software today. Everybody is doing it, and it fits nicely in with the general bloat (size, functionality, ++) of today's software. Nobody is working to keep things small, efficient, resource friendly and what have you. Sadly. :-/
 

sidetone

Daemon

Reaction score: 610
Messages: 1,419

While the person got major details wrong. It is crazy to need to reinstall LLVM for using a graphics driver with Xorg.

Some programs want a different version of LLVM. Which there should only be one needed. Firefox and Thunderbird are Gecko ports which require Rust. Rust is fine, but it's crazy, when the newest version is wanted. I use pkg to install Rust, and the latest available versions of LLVM. When building a program that wants the latest LLVM or Rust, and the wanted version isn't available from packages, I use portmaster with the -i option to forgo (re/-)building LLVM or Rust from ports.
 

facedebouc

Member

Reaction score: 5
Messages: 34

On my desktop computer I don't see those dependencies you are describing here.
I only have one llvm :
Code:
$ pkg info -x llvm
llvm80-8.0.1_4

and no rust.
It's an only packages installed system. I don't compile any ports.
My packages are built with ports-mgmt/poudriere for all my FreeBSD computers.
My repository is hosted on a VPS and it's great.
 

drhowarddrfine

Son of Beastie

Reaction score: 1,733
Messages: 3,814

It seems, then, that the OP is building a desktop from ports rather than using packages.
If the maintainers did not stick closely to upstream, then we'd hear complaints about how their software doesn't work on their system for all kinds of reasons.
 

kpedersen

Daemon

Reaction score: 1,262
Messages: 2,189

For most of my installs I generally just use packages and yet often end up with a considerable number of llvm versions (only slightly less than python installs). Most of them came from the xorg meta-package and dependencies due to I believe some of the AMD code generation stuff.

That said I have a fairly fresh install and only have 2 llvm packages installed suggesting that things have improved recently.
 
OP
D

dpecher

New Member

Reaction score: 9
Messages: 6

Updated my ports tree as I thought I may have been looking at old info but Xorg doesn't appear to depend on librsvg at all. So I'm still wondering why Rust would be a dependency of Xorg.

It comes in at compiling gobject-introspection, which in turn seems to have been adde through ticking the vmware-driver in the x11-drivers menu. I already paid attention not to add any features and leave everythingat default, but when running FreeBSD in a virtual machine, you sort of have to add the vmware graphics and mouse drivers, and that's when half the ports tree seems to be added, including llvm9 and rust. That's an instant 4 hour time penalty on an 8th gen i5.
 
OP
D

dpecher

New Member

Reaction score: 9
Messages: 6

It seems, then, that the OP is building a desktop from ports rather than using packages.
If the maintainers did not stick closely to upstream, then we'd hear complaints about how their software doesn't work on their system for all kinds of reasons.
Yes, I build the x11/xorg package from source, yes. But that seems to be quite effective in building half the port tree by the look of it.

I run FBSD11 and FBSD12 in VMs on an ESX rack and installing the binary packages can leave you with fairly outdated packages (for instance binary xorg install pulls llvm8, while building from source pulls llvm 10 and llvm 9). I use these machines as build servers for our CDE sources. That they work with llvm8, well we tested that half a year ago. What we need to know is whether they build against the latest versions, so regular rebuilds are neccessary. For instance, the latest Gcc10 wrecked our builds, which you wouldn't find out going by using binary pkgs. (This last topic is not specific to FreeBSD though. It mainly concerned NetBSD, and a load of Linuces).
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,783
Messages: 34,910

installing the binary packages can leave you with fairly outdated packages
They're just as up to date as everything else. Note that there's a difference between quarterly packages (updated once every three months) and latest (which follows the ports tree as close as possible). What you think is "old" is probably from the quarterly branches, switch your packages to latest.
 

shkhln

Daemon

Reaction score: 749
Messages: 1,826

This is a shame. Do we really need two versions of LLVM to compile editors/vim?

Do I really want to enable Wayland everywhere because someday someone might use it?
You don't have to enable anything if you build packages from source. "Minimal" defaults would mean less people able to use the official binary packages, which would lead to more people rebuilding LLVM, which in turn means more complaints.
 
Top