The llvm "saga"

Minbari

Minbari

Before you are gone "attack" me I'm gone tell you something: I'm a FreeBSD user since 5.2 and I love it; also FreeBSD is my main OS despite the fact that I also run several GNU/Linux instances on other machines, but having multiple version of the same software is unproductive. For eg. right now I have installed 5 instance of llvm (8-12). Why the "F" llvm is not modular or a "meta/main" port? It's a waste of space, resources and compiling time. And yes I do mix ports and packages because I want to have a bug free, updated and sometimes coustom system (don't worry I do the mix since 9.2 and never breaked my system, since I'm on quarterly pkg and ports). So my question is quite simple why don't we have a "main" llvm and a "devel" one instead of "n" version? Are we afford to maintain different versions of "X" software? Last time I checked there where a hand of mans trying to maintain the tree ports.

PS: Don't suggest me "alternate" methods like my own repo because I don't have the resources and the time to maintime another repository.
 
Probably because major version revisions break backwards compatibility and port compilation hasn't been checked or updated against newer versions.
 
I suggest to see which llvm versions on your system are shown to actually be required by something:
Code: 
pkg info -rx llvm
On my machine, the results are:
Code: 
 % pkg info -rx llvm
llvm10-10.0.1_4:
        mesa-dri-20.2.3_1

In the past, I've seen multiple llvm versions listed, and some of those showed that they were not required by anything else. I believe that in most cases those older versions are left around because newer version of things like mesa-dri require a new llvm version, and the older llvm doesn't get removed automatically.

I just use pkg delete to manually get rid of the llvm versions that aren't needed.
 
Looks like mesa-dri got updated to v10 in December. Spidermonkey is odd, since it seems to want something very specific out of LLVM v9.
 
The graphics/mesa-dri port specifically seems to request devel/llvm10. Then there is /usr/ports/Mk/bsd.default-versions.mk which defines a default llvm version to use, which is as of now:
Code: 
# Possible values: 60, 70, 80, 90, 10, 11, -devel (to be used when non-base compiler is required)
# Please give notice to the Graphics Team (x11@FreeBSD.org) in advance before
# bumping the LLVM version.
LLVM_DEFAULT?= 90
Firefox, thunderbird, etc use /usr/ports/Mk/bsd.gecko.mk which believes that default isn't good enough, so they use devel/llvm11 instead, unless the user has specifically overridden that default:
Code: 
# Ignore Mk/bsd.default-versions.mk but respect make.conf(5)
.if !defined(DEFAULT_VERSIONS) || ! ${DEFAULT_VERSIONS:Mllvm*}
LLVM_DEFAULT= 11 # bump if not latest release
.endif
So on a desktop system built from ports, chances are that you will likely end up having at least llvm 9, 10 and 11 installed. And yes, it is a mess.
 
This is a problem, and it probably runs people off. It's difficult to now say, FreeBSD is less bloated than Ubuntu, because of the compiler situation.

The way LLVM is implemented now is a nuisance to FreeBSD users who will stay here.


Checking if ports compile under a version of compiler, could help them at least reduce the versions of llvm required.
 
Applying restricted policies, regarding add/update ports in Ports Collection is one solution. But that's (probably) too much! and I doubt it's ever going to happen. It implies: we will have less ports.
 
I haven't noticed the LLVM version spam being too bad lately.
A year or so ago I used to install "big stuff" into a Jail and either run it from there or use unionfs to temporarily merge it into my /usr/local when I needed to run it. It was fairly hacky but did well to keep my machine clear of some mess.
 
  • Thanks
Reactions: a6h
kpedersen said:
I haven't noticed the LLVM version spam being too bad lately.
Click to expand...
Me neither. Although my anecdotal experience is a very bad one. My FreeBSD installation is minimal CLI-only, with very few packages. I even don't install vim-console. Just plain good old nvi!
 
Minbari said:
Before you are gone "attack" me I'm gone tell you something: I'm a FreeBSD user since 5.2 and I love it; also FreeBSD is my main OS despite the fact that I also run several GNU/Linux instances on other machines, but having multiple version of the same software is unproductive. For eg. right now I have installed 5 instance of llvm (8-12). Why the "F" llvm is not modular or a "meta/main" port? It's a waste of space, resources and compiling time. And yes I do mix ports and packages because I want to have a bug free, updated and sometimes coustom system (don't worry I do the mix since 9.2 and never breaked my system, since I'm on quarterly pkg and ports). So my question is quite simple why don't we have a "main" llvm and a "devel" one instead of "n" version? Are we afford to maintain different versions of "X" software? Last time I checked there where a hand of mans trying to maintain the tree ports.

PS: Don't suggest me "alternate" methods like my own repo because I don't have the resources and the time to maintime another repository.
Click to expand...
I hear you brother!
Let's be clear, this is not just a problem of llvm, it's gcc as well (less nowadays).

My opinion (yes, just that!) is that the ports tree is out of control and has been for a LONG time. I avoid using it or pkg as much as possible and prefer to build and change to avoid things like multiple versions of things like llvm. I will take the patches from the ports but that's about it.

I also realise it's infeasible for all users or even the OP to do this but it just points to my issue with the ports tree; one I've held for a long time (both personally and professionally).
 
GCC on FreeBSD has been fixed by a lot.

Before this LLVM mess, it actually was more capable and compiled in a comparable amount of time to LLVM/Clang. Years before, GCC took over 20 hours, because it pulled in dependency hell. GCC may have compiled in 2 to 4 hours after it was fixed.

Now ports demand a certain version of LLVM: having GCC won't be of help, because of this.
 
  • Thanks
Reactions: a6h
On my desktop machine which is entirely built/updated using ports, I made the experiment to set the default llvm version to 10, which is required by graphics/mesa-dri anyways. As a consequence devel/qt5-qdoc which would normally use the default llvm version (9) is now using llvm10 and firefox/thunderbird which would normally be built using llvm11 are now built with llvm10 also, thereby reducing the number of installed llvm versions to only devel/llvm10. So far I have not encountered any negative side effects.
 
OT:

To defend LLVM (just a little!), and also the problem of multiple versions of same shared library (so),
I have to compare FreeBSD to Windows (*)

1. It's fair to say, I've never been a GCC user (not literally)
2. I've directly jumped from MSVC to LLVM.
3. In windows, I was/am working on sound/waves/RF and sometimes MCU related programs (mostly legacy),

and to use/develop/debug such programs:
4. It's very common to have/installed more than:

I. 5 to 10 different versions of "Microsoft Visual C++ Redistributable".
II. 2 or 3 different versions of ".NET Framework" and/or ".NET core".
III. Same goes for different versions of MSVC, C++/CLI.
IV. And finally, often different versions of ATL and/or MFC.

5. I can't judge FreeBSD/LLVM by its own. Comparing to Windows/MSVC, I think FreeBSD/LLVM is in very good shape.
6. Of course It's not an ideal situation, and also there's the problem of ports/compilation (not always),
but there's no such thing as ideal situation.


[EDIT]

(*) Why not to compare FreeBSD to GNU/Linux:
I. Comparisons have to be practical,
II. I have to compare two different working environment in a real world/work scenario.
III. I don't use Linux.
 
mickey said:
On my desktop machine which is entirely built/updated using ports, I made the experiment to set the default llvm version to 10, which is required by graphics/mesa-dri anyways. As a consequence devel/qt5-qdoc which would normally use the default llvm version (9) is now using llvm10 and firefox/thunderbird which would normally be built using llvm11 are now built with llvm10 also, thereby reducing the number of installed llvm versions to only devel/llvm10. So far I have not encountered any negative side effects.
Click to expand...
And you won't. This is the reason why requiring multiple versions is part of the ultimately broken port system (in this regard).

Unless some application is using some exotic, llvm-only compiler version specific option (which they aren't), then there is ABSOLUTELY no reason for all these multiple versions. It even goes so far as libraries and other utilities where a specific version is required and some packages require 2 versions of the same library/runtime/application to compile. It's madness, but that's an even bigger issue.
 
vigole said:
OT:

To defend LLVM (just a little!), and also the problem of multiple versions of same shared library (so),
I have to compare FreeBSD to Windows (*)

1. It's fair to say, I've never been a GCC user (not literally)
2. I've directly jumped from MSVC to LLVM.
3. In windows, I was/am working on sound/waves/RF and sometimes MCU related programs (mostly legacy),

and to use/develop/debug such programs:
4. It's very common to have/installed more than:

I. 5 to 10 different versions of "Microsoft Visual C++ Redistributable".
II. 2 or 3 different versions of ".NET Framework" and/or ".NET core".
III. Same goes or different version of MSVC, C++/CLI.
IV. And finally, often different versions of ATL and/or MFC.

5. I can't judge FreeBSD/LLVM by its own. Comparing to Windows/MSVC, I think FreeBSD/LLVM is in very good shape.
6. Of course It's not an ideal situation, and also there's the problem of ports/compilation (not always),
but there's no such thing as ideal situation.


[EDIT]

(*) Why not to compare FreeBSD to GNU/Linux:
I. Comparisons have to be practical,
II. I have to compare two different working environment in a real world/work scenario.
III. I don't use Linux.
Click to expand...

Your comparison has some merit, as you mention run-time libraries. This is, however, not the same as perhaps requiring (I am not sure of the nomenclature of Windows compilers) Visual studio 2015,2017,2019 to compile 1 or more program(s).

Multiple run-time libraries are inevitable, and they should be better controlled so they too don't end up with 10s of version, but that's another issue, I think.

I guess, ultimately, the comparison is hard to make; they're totally different close v open source systems.
 
  • Thanks
Reactions: a6h
mark_j said:
This is, however, not the same as perhaps requiring (I am not sure of the nomenclature of Windows compilers) Visual studio 2015,2017,2019 to compile 1 or more program(s).
Click to expand...
You're absolutely correct.

I have to admit my comparison was a very lame one. I don't think it even counts as a valid comparison! But the point was:
The problem of multiple compiler/library/etc is a universal problem among probably all OSs. I'm not sure any OS solve that completely/correctly.
It's very annoying, but what we can do! Compilation is very time consuming, especially when you don't have proper hardware/resource (like me!).
But I can't see any easy solution for this problem. Restricting committed ports are one solution, but that's another story.
 
mark_j said:
And you won't. This is the reason why requiring multiple versions is part of the ultimately broken port system (in this regard).

Unless some application is using some exotic, llvm-only compiler version specific option (which they aren't), then there is ABSOLUTELY no reason for all these multiple versions. It even goes so far as libraries and other utilities where a specific version is required and some packages require 2 versions of the same library/runtime/application to compile
Click to expand...
I don't entirely disagree with this, but a "FreeBSD" answer to these concerns are:

1. Use packages instead
2. If you have to compile, use a build system where it won't concern you on your desktop and then yank those packages over the network
3. This is a maintainer issue, not a "port system" issue
 
msplsh said:
3. This is a maintainer issue, not a "port system" issue
Click to expand...
A maintainer issue is a subset of a ports system issue. Sometimes problems are more intertwined or more closely related. LLVM version is also set by macros in the Mk/ directory.
 
I agree the ports tree has turned into a bloated dependency structure. I see ports every now and then on a minor update suddenly require a new dependency, often this is because a new option gets added to the port and its "enabled" by default, sometimes it happens without even an option been added, there seems to be favouring feature bloat over streamlined. The LLVM issue from what has been described seems to be down to a lack of effort to try and ensure there is minimal versions been depended on via ports, the solution isnt to remove the choice from ports but rather to streamline the ports that call for the dependencies.

I think last year I once installed a port that pulled in nearly 20 dependencies, none of them perl modules. I disabled an option and it was down to 2 dependencies.
 
vigole said:
You're absolutely correct.

I have to admit my comparison was a very lame one. I don't think it even counts as a valid comparison! But the point was:
The problem of multiple compiler/library/etc is a universal problem among probably all OSs. I'm not sure any OS solve that completely/correctly.
It's very annoying, but what we can do! Compilation is very time consuming, especially when you don't have proper hardware/resource (like me!).
But I can't see any easy solution for this problem. Restricting committed ports are one solution, but that's another story.
Click to expand...
You bring up a good point though, and that's hardware/resources. Most people who run ARM-based small systems, like Raspberry Pis are faced with this issue. Multiple compilers required not only take up space but time to build. Streamlining it to one version is almost imperative.

So, you're stuck with packages which shoe-horn you into using X option as compiled/built by the package system.

I think it's probably an unsolvable problem. As the number of ports grow so does the chances of having multiple versions of the same compiler. I just think more co-ordination within the ports sphere could solve this issue.
 
  • Thanks
Reactions: a6h
chrcol said:
I agree the ports tree has turned into a bloated dependency structure. I see ports every now and then on a minor update suddenly require a new dependency, often this is because a new option gets added to the port and its "enabled" by default, sometimes it happens without even an option been added, there seems to be favouring feature bloat over streamlined. The LLVM issue from what has been described seems to be down to a lack of effort to try and ensure there is minimal versions been depended on via ports, the solution isnt to remove the choice from ports but rather to streamline the ports that call for the dependencies.
Click to expand...
Precisely.
 
It helps a little to use static linking. You get off the constant upgrade train for a while. We use a lot of large open source projects where we want a small slice of functionality, so often times the
creeping incompatibilities are irrelevant.

its amazing the language c++ has become because of the constant changes. we are lucky to have it. but it breaks things.
 
You must log in or register to reply here.
Back
Top