Ports needs structure

[sidetone]

A lot of circular dependencies have been removed from FreeBSD's ports tree, which is good. There are still a lot of circular categories in the port's options. For instance graphical ports will call audio or internet dependencies, then they will call on graphical dependencies, and repeatedly vice versa.

I understand that's how many of the programs were designed, and this was simply brought over. This happens a lot with gtk, and ffmpeg which want to reinstall everything all over and redundantly, both overlapping each other and the program's specific dependencies on audio and graphics. It may take a lot of work to organize the dependencies by category/type. Another example is when Alsa has to repeat OSS, when if Alsa has something that OSS is missing, why can't Alsa be trimmed down, and built with OSS as the base. Most BSD made dependencies should be on the base layer, unless a GPL licensing restriction prevents that.

I'm pointing out, that making structural guidances, and passively fixing it when this is found, instead of proactively overhauling the ports tree may be a more manageable way to fix inefficiencies.

 

[zirias@]

I doubt that these two goals:

• provide all options of upstream source in ports while modifying as little as possible
• prevent all possible circular dependencies

can ever be met at the same time. Focusing on the second would heavily degrade the flexibility of the ports system, so I'd say if you build packages yourself, you can be expected to create your personal dependency tree from a dependency forest yourself, too.

That said, there should ideally NEVER be a circular dependency with default build options.

 

[kpa]

You can't control third party software well enough to avoid potential circular dependencies. The design choices are made by the upstream developers that are not thinking how their choices affect the FreeBSD ports.

 

[sidetone]

It can't be 100% done, and some does require reporting bugs to third party developers that often have fixed programs that are outside of FreeBSD's scope. To some extent it has been done here, for instance GCC has been trimmed down with removals of dependencies not useful for FreeBSD. Bugs are often reported on ports that don't build on Clang/LLVM. Before GCC was fixed it used to take over 12 hours to compile, then break. Also, when that was replaced with Clang (built in the base, but still more efficient), those programs compile times dropped to less than 5 minutes and had a smoother running program. Circular dependencies are supposed to be reported. What can be controlled, is how the options and defaults are made in the Makefiles. I think ffmpeg and gtk can be manipulated as little as, ffmpeg deals with audio and codecs while, gtk only deals with graphics. OSS is a BSD type of program, which I understand that some programs are incompatible with, so shouldn't the required ALSA be built partially with a basis on top of OSS.

 

[zirias@]

Ok, let's put a practical example here:

Assume there is an application building toolkit, consisting of quite a few libraries, one of them is for GUI, one is for printing, and so on. Let's say libsuperkitgui.so is for GUI, libsuperkitprint.so is for printing.

Then there is a printing system, with an optional management tool using the GUI from the "superkit". So, for building "fantasticprint", you CAN build a fantasticprintmanager that links to libsuperkitgui.so.

Further assume our "superkit" wants to support printing using "fantasticprint" as an option. If selected, libsuperkitprint.so would reference libfantasticprint.so.

Now imagine you select both options at the same time. The respective ports would end up with a circular dependency. But there won't be any circular dependency WRT the installed binaries and shared objects, so, nothing to break a running system.

What I want to show with this example: ports aren't fine-grained enough to really tell that a circular dependency is a problem. I don't think it would be a great idea to tell the user that he either can't have fantasticprint use superkit, or vice versa. I'd be happy when just default build options don't expose any circular dependencies.

 

[sidetone]

It should be the dependency for both, but in the simplest way possible. For instance, it be a least common factor by category, and not calling on hundreds of other programs. Calling a few unneeded programs is unavoidable, and that's fine. Both should call it, but it shouldn't be where that one can go on and call on audio, graphical, network, etc other dependencies. I think libraries have more maneuverability to be called on by different types of programs, but they should be used as the least common factor, or as long as they don't call the full programs which call on clusters of unrelated dependencies.

 

[sidetone]

It wouldn't make sense to have rigid rules about ports, but instead an improvement of guidelines that act as suggestions.

The programs' way is set by the developers and not at FreeBSD. Much of how it compiles is up to them, but FreeBSD does have some control over how they are compiled. Look at how 1 program will have different ports for it at FreeBSD, with a different suffix for different purposes. When a port complexity is a big deal, it gets reported as a bug, and they fix that upstream. This is only useful for huge problems, and not for small ones.

Here's an example of an upstream adjustment for a major issue... When FreeBSD switched to LLVM/Clang in its base system, some programs still required GCC. What happened was, that GCC tried to reinstall everything that FreeBSD already had in its base system, but wasn't used or needed. This goes from ports needed for mounting, graphics, audio, duplicates of what the base system had. This install carried on a compile for 16 hours. The program needed on a handful of ports, which took 5 minutes of compiling time, but pulling in that 1 dependency made a difference in compiling 16 hours of bloat. Switching over to CLANG/LLVM showed this. Despite that, GCC dealt with that issue as a bug, and they've improved GCC by removing the need for it to compile 16 hours of bloat for no reason. Now GCC compiles a lot better, it takes maybe 30 minutes to an hour to compile, because that duplicity wasn't needed when FreeBSD was already a full operating system, not needing a duplicate operating system. This is an actual example, while it is extreme, it is important enough where a bug report adjusted an upstream program. Before, it would compile for hours, usually break, and the compiled program would run a lot slower. I understand that LLVM and Clang can be compiled, or are in the base system. With that upstream fix, LLVM/Clang's compilation time is within the ballpark of GCC's compile time on FreeBSD.

A lot of circular dependencies have been fixed. The concept I'm bringing up is dependencies by category. qt4 deals with it a lot better, but some of its dependencies try to be a full solution to everything. For now this is fine. I see ffmpeg was a bad example, because it does focus on codecs. Still many port compiles will go without order, from a port that deals with codecs, to one with graphics, back and forth. Shouldn't it be structured somewhat, to where it compiles most ports pertaining to graphics, then most ports pertaining etc, and not a tangleweb.

FreeBSD has some control, for instance of how different variants of the same ports will be compiled. Libraries are often needed by many different ports, and often they are the least common factor, and often only need to pull in libraries only, not full ports. I think there can be loose guidelines to improve how ports are made.