Deprecating base system ftpd

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

kpedersen, you only proved that you have NO idea how pkg-base works. It doesn't change *anything* in the structure of base, neither in the source nor in the installed version. It just enables to build individual binary packages. See also my edit above.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

FreeBSD isn’t POSIX / SUS compliant anyway. ;)
No but it certainly doesn't seem good to be moving in the wrong direction either.
I’m just saying that it would be useful to have an option to not install it if you don’t want it.
If you want to rice with things, then

Code:
rm /usr/bin/clang
rm /bin/ed

And just remember to mention that on the mailing lists if you ever run into issues and need troubleshooting.

Whats next? Split development libraries from packages? aka zlib, zlib-dev
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Whats next? Split development libraries from packages? aka zlib, zlib-dev
Given some packages pull in a whole gcc10, just to have a few runtime libraries, a mechanism to build more than one package from one port is desperately needed.*) But that's a different topic.

*) and FWIW, I've seen this issue discussed on IRC and mailing lists.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

Given some packages pull in a whole gcc10, just to have a few runtime libraries, a mechanism to build more than one package from one port is desperately needed. But that's a different topic.
Possibly a different topic. But I am again of a different idea when it comes to packages, less is better. I.e a gtk+ package should bundle all the gtk+ cruft such as pango, gobject, gconf into one package called gtk+. They never get used individually anyway. This is just modular monolith GNU crap.

As for the pkg-base, I am not entirely certain I have misunderstood the design. Just because programs are grouped into larger meta-packages with a different name doesn't mean they wont be fragmented and split at a later date.

Are you ever going to install the i.e compiler packages and not the runtime packages? What is the point of modularising them when a base is a monolithic concept?
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Possibly a different topic. But I am again of a different idea when it comes to packages, less is better. I.e a gtk+ package should bundle all the gtk+ cruft such as pango, gobject, gconf into one package called gtk+. They never get used individually anyway.
Except when they are, which is especially true for a few libraries out of a huge package with everything. The assumption that the machine running the software is also the build machine is flawed.
As for the pkg-base, I am not entirely certain I have misunderstood the design. Just because programs are grouped into larger meta-packages with a different name doesn't mean they wont be fragmented and split at a later date.
Then again, from what you write here, I'm pretty sure you misunderstand the concept. I'll try again: nothing will change about base being self-contained, one repository, developed and release-engineered as a whole. And then, infrastructure for building packages is already there. Once this is officially used, all that will change is instead of distributing one huge base.txz, distribute a lot of small .txz packages with (together) exactly the same content. This just means you don't have to compile yourself any more in order to choose to NOT install some component. And that's it…

(I guess one "blocker" for pkg-base to go "live" could be that you need to revisit ALL the ports, so they specify what they need from base. I'd even consider that a valid argument against it, as it's more work for port maintainers – although I think it's manageable once initially started…)

Are you ever going to install the i.e compiler packages and not the runtime packages? What is the point?
The usecase is the OTHER way around. Software compiled with GCC needs GCC runtime libraries. This is an extreme example, but one that's relevant for many packages. Just to run a software compiled with GCC, you need to install the whole huge GCC package to have the few libraries available.
See here: https://cgit.freebsd.org/ports/tree/Mk/bsd.gcc.mk#n148
This is added except for GCC-based architectures.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

Except when they are, which is especially true for a few libraries out of a huge package with everything. The assumption that the machine running the software is also the build machine is flawed.
So for the few gcc projects I had, to eliminate the dependency on gcc package at runtime, I just statically link the runtime in. Yes, not available for everyone but clang *is* the FreeBSD system compiler. If people go against that, that is a little bit their problem ;)

If you extract all those gtk+ packages, it comes to about ~5 megs. The package meta-data from all those packages in cache probably takes up more space than the packages themselves. Gtk+ is one thing.

Once this is officially used, all that will change is instead of distributing one huge base.txz, distribute a lot of small .txz packages with (together) exactly the same content. This just means you don't have to compile yourself any more in order to choose to NOT install some component. And that's it…
Thats fine. So long as you can cat those ratty small files together and rename it base.txz I am happy. Otherwise, why are they fragmenting those files?

What you are saying is that if they were made into .deb files and installed via apt-get it would work the same way right?
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

So for the few gcc projects I had, to eliminate the dependency on gcc package at runtime, I just statically link the runtime in. Yes, not available for everyone but clang *is* the FreeBSD system compiler. If people go against that, that is a little bit their problem ;)
So, it's your fault as an end user wanting to run a software that only compiles with GCC? How could you be so stupid wanting to use it? Then live with tons of "crap" installed you never need.
Thats fine. So long as you can cat those ratty small files together and rename it base.txz I am happy. Otherwise, why are they fragmenting those files?
I answered this very question twice. Calling it "fragmentation" is already (at best!) misleading. Do you have any argument for forcing anyone to always install the whole base, or otherwise compile himself? How does it hurt you?
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

The usecase is the OTHER way around. Software compiled with GCC needs GCC runtime libraries.
Yes, I know. I was being annoying to try to show that it is slightly of limited use.

So OpenBSD does a little bit similar with their base.tgz and comp.tgz packages. However they basically also say if you don't install comp.tgz (even on a server) you are being weird and you are on your own.

Ricing is something that should not be normalized or you will just end up with crazy fragmentation with users doing mad things.

So, it's your fault as an end user wanting to run a software that only compiles with GCC? How could you be so stupid wanting to use it? Then live with tons of "crap" installed you never need.
Or get someone to fix that broken crap so it compiles with Clang. We do this for MSVC++ centric code all the time right? It is broken and needs fixing.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

More strawman nonsense. It's just about not installing stuff you don't need.
So will the installer also provide a list of kernel modules / firmware the user probably wont need and they can pluck them away?

Honestly I see what you mean if we were talking about serious size but for the sake of ~10 megs it is *not* a good use of time.
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

I don't care how exactly an installer making use of pkg-base looks like. Most probably, it will just install the whole base anyways by default. The relevant thing is you can remove things you don't want/need (even later).

Again, you can already right now, but you have to compile yourself to do so. My reason for leaving out SSH is I always use the one from ports, linked against libressl. I don't want to have a second unused variant installed, and this is also for minimizing the risk that it is, accidentally, used. For other stuff I leave out, the motivation is more to save unnecessary build time than disk space, but again: How does it hurt you when users of binary releases have this freedom as well?
 

olli@

Daemon
Developer

Reaction score: 1,205
Messages: 1,108

No but it certainly doesn't seem good to be moving in the wrong direction either.
Are you misunderstanding me on purpose? Again, I’m not saying that FreeBSD should ship without clang.

If you want to rice with things, then

Code:
rm /usr/bin/clang
rm /bin/ed
No, the correct approach would be something like pkg delete ed (similarly for clang) so its (non-)presence is properly recorded in the package database, so other things that depend on it can react accordingly.

And just remember to mention that on the mailing lists if you ever run into issues and need troubleshooting.
It’s unlikely that there will be any issues. What kind of issues would you expect? If a 3rd-party package requires ed, it has a dependency recorded and pulls in the ed package if needed, like with any other dependency.

Whats next? Split development libraries from packages? aka zlib, zlib-dev
We already do that for quite some time.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

Are you misunderstanding me on purpose? Again, I’m not saying that FreeBSD should ship without clang.
I am sure there are many who think that it should. And now (or in the future) we will have to deal with their broken installs.

What I am saying is that we should disallow:

# pkg remove compiler

Not because I am mean but because it would generate a mess of different installs. Freedom is great for those who don't need to provide support to others.
It’s unlikely that there will be any issues. What kind of issues would you expect? If a 3rd-party package requires ed, it has a dependency recorded and pulls in the ed package if needed, like with any other dependency.
For deployment it is nice to have a known base. It means in some cases, you don't even need to make a port or package. The fact that this base can now be minced up reduces the guarantee and either a big list of packages needs to be listed as a requirement or I need to make a formal port.

Consider if on Windows it was possible to randomly uninstall conhost or user32.dll. You would expect all 3rd party software to gracefully handle that? FreeBSD really isn't much different. Known configurations are nice.

We already do that for quite some time.
That is news to me. I'm running FreeBSD where if I install i.e zlib, I get the lib and includes. What OS are you running?
 

sidetone

Daemon

Reaction score: 702
Messages: 1,539

When I looked into pkg base, it looked like I had to compile my world to get it. It seemed to be useful for when someone wants to distribute that to similar machines. Maybe future use was intended what I thought it was, for base packages to be direct packages from FreeBSD, without the user needing to recompile world.

If that's the case, the install wouldn't be just the kernel. It would have many of the tools of a small system rescue CD.

I see LLVM and Clang staying in it, unless they're base packages. Everything is in C. Rust is a cool idea, but it didn't turn out as well as I thought it did for a system that's written in C. Rust makes sense for being in Redox. On FreeBSD, Rust programs run faster than I thought possible, but they have a parallel package build to FreeBSD's ports. I don't know if it could ever turn out, that there would be a slim FreeBSD in C without a Clang compiler, then there be two directions to go, to install Rust and only have Rust programs in packages (optionally bypassing ports), or to install Clang and go with ports, or have both. But then, you'd want Clang in base for everyone to at least recompile the kernel in C, if not everything else in base, or base packages.

The benefit of having a base package, would be, we have to wait for a new release to get a newer LLVM and Clang, when ports already require the latest versions. This would be a benefit of not having LLVM and Clang in base, and having it in base packages. There was an implication of having LLVM in base, but not Clang, but some ports may require that both be upgraded.

If base packages work like that, it would be good, as that system can be much more better than ports, and without Linuxism dependencies. For instance, base packages relying on bmake instead of gmake, or having OpenBSD's, NetBSD's or DragonFly's implementation of programs instead of GNU's or FreeDesktop's. Then, the rest in ports including Linuxisms can sit on top of a more efficient system, which ports will itself be structured more efficiently as a result.

Linux emulation would be better or more appealing to more users too, For emulation, I don't understand why there are two sets of program dependencies in ports: the one for everyone and the Linux emulated version. Shouldn't the emulation layer use everything it can from the ports and base that are used for everyone, and only have additions for what's lacking from the one FreeBSD offers in ports for everyone?
 

ralphbsz

Son of Beastie

Reaction score: 1,978
Messages: 2,942

And clang – well, why shouldn’t it be optional?
Some of us are old enough to remember what happened when Sun for the first time shipped SunOS without a compiler, and then required paying of extra license fees to get a compiler. Actually, the correct term is not "remember", but "still feel the pain and are still mad". So removing the compiler from the most basic OS is likely to create a strongly emotional response.

In fact, when I install FreeBSD on a space-constrained machine, I remove the compiler tool chain, static libraries, include files and several other things. It would make sense to be able to de-select these in the installer if you don’t need them.
Completely agree. I'm running a few RPi0, and on those I would never think of doing a major compile, and disk space is at a premium, so not having the compiler and all the development libraries is sensible.

So I just gave a strong argument for keeping the compiler as mandatory, and one for making it optional. Resolving such contradictions is hard. Similar sets of arguments can be made for and against removing ftpd.

Ultimately, this will be a decision that's based on judgement. Part of that judgement is knowing history, knowing the personalities involved, knowing how people feel. For example: Berkeley without a campanile is not Berkeley. Should the campanile fall down in the next earthquake, it would need to be rebuilt. Not because it is useful, but because it defines what "Berkeley" means, together with other things (like Sproul plaza and Zellerbach hall). Similar arguments go for the Hoover tower, quad and memorial church and Stanford. And in the same fashion as a Usenix conference without Kirk and Eric is not a Usenix conference, a BSD without UFS and Sendmail is not BSD. That's not a technical argument (ZFS works just fine, and most people's need for a MTA is better met by other software), but a psychological or political one.
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Not because I am mean but because it would generate a mess of different installs.
So, what problems exactly would you expect? We already have working dependency mechanisms in ports and packages.
For deployment it is nice to have a known base. It means in some cases, you don't even need to make a port or package. The fact that this base can now be minced up reduces the guarantee and either a big list of packages needs to be listed as a requirement or I need to make a formal port.
Did you ever create a port? Adding a dependency isn't a huge thing to do, and they work transitively.
Consider if on Windows it was possible to randomly uninstall conhost or user32.dll. You would expect all 3rd party software to gracefully handle that? FreeBSD really isn't much different. Known configurations are nice.
So, right now, FreeBSD is just like Windows regarding the base system, but unlike Windows has a well-working package management. Then your argument is, it should still keep the drawbacks of Windows?

I still didn't see a single valid argument what the problem would be. Maybe I have to repeat that a few more times: I already have a long list of WITHOUT_XXX knobs for my base. It doesn't create any problem.

The benefit of having a base package, would be, we have to wait for a new release to get a newer LLVM and Clang, when ports already require the latest versions. This would be a benefit of not having LLVM and Clang in base, and having it in base packages. There was an implication of having LLVM in base, but not Clang, but some ports may require that both be upgraded.
Not sure if that's what you're implying here, but individually upgrading packages in base isn't the point of pkg-base and shouldn't (and, I'm pretty sure, WON'T) be supported.
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Some of us are old enough to remember what happened when Sun for the first time shipped SunOS without a compiler, and then required paying of extra license fees to get a compiler. Actually, the correct term is not "remember", but "still feel the pain and are still mad". So removing the compiler from the most basic OS is likely to create a strongly emotional response.
You're really beating this strawman to death.
Making installation default, but optional, is something completely different than removing it from base.
 

sidetone

Daemon

Reaction score: 702
Messages: 1,539

Some of us are old enough to remember what happened when Sun for the first time shipped SunOS without a compiler, and then required paying of extra license fees to get a compiler. Actually, the correct term is not "remember", but "still feel the pain and are still mad". So removing the compiler from the most basic OS is likely to create a strongly emotional response.
I doubt anything like this will happen. It's more of a fear on something that happened from somewhere else. That wouldn't fit FreeBSD's business model anyway. Also, the base has a set standard to use certain types of licenses, pkg base will be like this too or more or less like this.

It would be cool, if the opensource compiler is always available in pkg base, then allow others especially companies to choose to use that or use a proprietary one. Of course, a BSD-like license would apply, so it would remain a choice.

If FreeBSD went that way of no longer offering the compiler in base packages or base, it would fall apart after losing lots of followers, and it would go against what FreeBSD was based on. It wouldn't be going with what everyone knows FreeBSD to be.


Not sure if that's what you're implying here, but individually upgrading packages in base isn't the point of pkg-base and shouldn't (and, I'm pretty sure, WON'T) be supported.

Maybe not. The only useful utility for that to happen would be for when LLVM or Clang or something like that needs to be updated before the next release comes about. If base pkg is set to remain at one version, which makes sense for most programs, it would be nice if the compiler and toolchain is the exception.

They have to work on multiple versions of the compiler as it is, and wait for new releases. It would be better, if they worked on one, and let this update as time goes on, instead of having incompatibilities from building different ports.
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Well, after years of work on it (and, maybe, more years to come), some day someone will have to admit: pkg-base, killed by FUD…

IIRC, pkg-base was worked on even before 12.0-RELEASE. It was never about doing that Linux distro crap of individual source packages for everything, but always just about building individual binary packages from the one large source tree. All it does is give the option to NOT install something – without having to build yourself.
 

kpedersen

Daemon

Reaction score: 1,568
Messages: 2,423

Completely agree. I'm running a few RPi0, and on those I would never think of doing a major compile, and disk space is at a premium, so not having the compiler and all the development libraries is sensible.
I'm not sure but don't a number of Perl, Python, Node.js dependencies compile up their native binaries during install / first use. Just installing something from PIP / CPAN will likely fail unless you have a C compiler installed.

Well, after years of work on it (and, maybe, more years to come), some day someone will have to admit: pkg-base, killed by FUD…
I wish. Sounds like it is going ahead. I personally think it is a bad decision. That said, I am still not impressed by the entirety of pkgng which seems to be an enabler of this.

Making installation default, but optional, is something completely different than removing it from base.
Really its not. At least if it is removed you know where it stands and you can make more educated guesses about peoples setups. Either way you will now have to explicitly state "make sure to have ABC installed"

Did you ever create a port? Adding a dependency isn't a huge thing to do, and they work transitively.
Do you add a dependency on xz to extract the package itself? Can I perhaps just add one dependency called "base"? This whole situation is very disappointing.

All it does is give the option to NOT install something – without having to build yourself.
Time will tell. It sounds like a clusterfsck to me ;)
 

Zirias

Daemon

Reaction score: 1,070
Messages: 1,933

Do you add a dependency on xz to extract the package itself?
Yes, see https://cgit.freebsd.org/ports/tree/Mk/Uses/tar.mk#n11
It's as simple as USES= tar:xz.
Can I perhaps just add one dependency called "base"? This whole situation is very disappointing.
I'm still convinced you just don't understand how it works. For the example above, nothing would have to change in the port. USES= tar:xz would just automatically add that dependency on the required base package.
 

Jose

Aspiring Daemon

Reaction score: 685
Messages: 790

Some of us are old enough to remember what happened when Sun for the first time shipped SunOS without a compiler, and then required paying of extra license fees to get a compiler. Actually, the correct term is not "remember", but "still feel the pain and are still mad". So removing the compiler from the most basic OS is likely to create a strongly emotional response.
Sunsite! I forget the full URL, but that's where you had to go to make your shiny new Sun box useful.
 

Jose

Aspiring Daemon

Reaction score: 685
Messages: 790

In all fairness, I am fine with ralphbsz's suggestion. Even if all the packages in base are specifically tailored for him, I would rather that even for my use-case. So long as base *is* a base rather than a scatty ever changing collection of packages.
Devil's advocate argument. It already is. It's just hidden from you by the fact that a bunch of sources are downloaded and checked into the src repo.

Is it better to do it that way or to have a bunch of packages that ultimately do the same thing? I honestly don't know. I see the value of being able to remove the compiler toolchain on memory-, storage-, and CPU- constrained environments, but I've also felt the frustration of not having a compiler installed by default.

My weakly held opinion is that it is better to have something like the src repository that is compiled, integrated, and tested as a unit to provide a sane foundation for doing computer things. I see the value of tweaking what's in that src repository to suit your needs too.

Addressing the meta-argument - yes, we're bikeshedding here, but it's fun bikeshedding, dammit!
 

PMc

Daemon

Reaction score: 553
Messages: 1,124

Do we get gitlite (or igitt) in base?
From how it appears to me, we cannot build a kernel without, because we do not get a version string into uname. But then, git requires 89 prerequiste ports - and I am not really into installing all these into a bhyve before building a kernel...
 

mark_j

Aspiring Daemon

Reaction score: 484
Messages: 883

Do we get gitlite (or igitt) in base?
From how it appears to me, we cannot build a kernel without, because we do not get a version string into uname. But then, git requires 89 prerequiste ports - and I am not really into installing all these into a bhyve before building a kernel...
Don't know them but git won't because of the license, unless core reverse their approach on gpl.
 
Top