Ports need stability

Hi,

I thought of telling a tale of woe about how I waited two years to upgrade to 8.0, and then libpng smashed the ports tree in the first week, and hurt me so bad... Instead, I'd like to appeal to the core team, because the problem is so obvious.

A naked o/s can't do much without applications. Ports are therefore critical to the success of FreeBSD. For a stable production environment, ports also need to be stable.

Ports are not stable. On the contrary, the ports tree is simply out of control.

Letting the hackers (traditional meaning) into the production code base is like letting the fox into the hen house.

I mean no disrespect to the people who labour there, but I can't think of a time in the last few years (probably since KDE and Gnome started) where the ports tree has not been causing me (semi-continuous) grief.

I don't want the latest features. I do want my systems to run reliably!

May we PLEASE have a stable version of ports that is not broken almost daily?!

--
Phil
 
Hi Phil,

Please, don't take me as being combative or anything, that is not my intent.

I have a production server I manage with FreeBSD (right now 7.1, but we're evaluating an upgrade path to 7.3 before January when 7.1 loses security support). 3rd party apps are built/installed from ports system.

I've only been bitten once by something, and it was when I blindly upgraded dovecot. Turns out I needed to make some changes in my config file before it would start back up again. Other than that, I've had no problems with the ports tree.

I routinely go a few weeks between upgrading any of the software to it's "later version" (unless of course there is some security issue which requires an immediate upgrade). This policy has worked fairly well. Nothing says you have to update a port every time someone commits a change to the ports tree.

I quite like the fact that ports and base are so separated. For example, if I were using Debian, and let's say I was running an older version (let's say 4.0 for the sake of argument), all the 3rd party apps would be tied to that release. Now, while I understand there is a good reason to do something like this, I much prefer to be able to run the latest (if I prefer) version of package X while keeping my base a few versions back from the latest and greatest. This is just my personal preference.

Here I've gone an rambled on, I'm sorry. Needless to say, the ports tree can be as stable or as broken as you want it. Using csup, you can even get an older ports tree if you wish. You don't have to have the latest and greatest, which may or may not be broken.
 
Why do you update installed ports then if you don't want new features and only want stability (except for security fixes)?
 
Hi,

I don't care to justify my need to occasionally install new ports on a production system. It happens. That is enough.

The current ports tree has been compromised for weeks because of just one update (to libpng). Make no mistake, the ports have been seriously broken. Yes, the ports maintainers are labouring valiantly to fix it. However...

I am suggesting that FreeBSD needs to embrace the principle of having a STABLE version of the ports tree guaranteed to be free of incompatible dependencies. This is hardly a revolutionary suggestion, as the principle is already applied to the o/s and core utilities.

Cheers,

--
Phil
 
gpw928 said:
I thought of telling a tale of woe about how I waited two years to upgrade to 8.0, and then libpng smashed the ports tree in the first week, and hurt me so bad... Instead, I'd like to appeal to the core team, because the problem is so obvious.

A naked o/s can't do much without applications. Ports are therefore critical to the success of FreeBSD. For a stable production environment, ports also need to be stable.

Ports are not stable. On the contrary, the ports tree is simply out of control.

Letting the hackers (traditional meaning) into the production code base is like letting the fox into the hen house.

I mean no disrespect to the people who labour there, but I can't think of a time in the last few years (probably since KDE and Gnome started) where the ports tree has not been causing me (semi-continuous) grief.

I don't want the latest features. I do want my systems to run reliably!

May we PLEASE have a stable version of ports that is not broken almost daily?!

If you don't update the ports tree, you have a perfectly stable and unchanging tree to install apps from. ;)

And, if you don't want the hassle of installing from an updated ports tree, then use packages.

Switching to "a stable ports tree" means we end up with the "debian way" where you are stuck the same versions of software for 3 years, while the rest of the world uses the fun stuff.

The ports tree is what you make of it. Just because there are updates to it daily, doesn't mean you should be updating everything on a daily basis.
 
gpw928 said:
I am suggesting that FreeBSD needs to embrace the principle of having a STABLE version of the ports tree guaranteed to be free of incompatible dependencies. This is hardly a revolutionary suggestion, as the principle is already applied to the o/s and core utilities.

;) And, what are you offering to help bring that to fruition? ;)
 
Hi Freddie,

If you don't update the ports tree, then the tarballs you need will eventually become unavailable. BTDT.

I suggested a "stable *version* of ports". This does not preclude the current arangement. It is an additional option which eliminates the bleeding edge breakages where different ports have incompatible dependencies.

--
Phil
 
phoenix said:
And, if you don't want the hassle of installing from an updated ports tree, then use packages.


... and stay with unsecure/valuable system becase pacakges were built on 'RELEASE day' and after that time some security holes have been found.

You can of course rebuilt them, but You will have to also rebuilt dependencies, which means broken other packages that now wants to use jpeg-8 instead of jpeg-7 (en example), so in the end You will have to rebuild ALMOST ALL installed applitacions, just to fix one or two programs that have holes:

Code:
% [color="Blue"][B]portaudit -a[/B][/color]
Affected package: curl-7.19.6_1
Type of problem: curl -- libcurl buffer overflow vulnerability.
Reference: <http://portaudit.FreeBSD.org/c8c31c41-49ed-11df-83fb-0015587e2cc1.html>

Affected package: gd-2.0.35_1,1
Type of problem: gd -- '_gdGetColors' remote buffer overflow vulnerability.
Reference: <http://portaudit.FreeBSD.org/4e8344a3-ca52-11de-8ee8-00215c6a37bb.html>

Affected package: linux-f10-pango-1.22.3
Type of problem: pango -- integer overflow.
Reference: <http://portaudit.FreeBSD.org/4b172278-3f46-11de-becb-001cc0377035.html>

3 problem(s) in your installed packages found.

You are advised to update or deinstall the affected package(s) immediately.



@gpw928

+1
 
There is definitely a problem with the ports approach which is why I think there needs to be a more comprehensive approach to packages which keeps packages up to date with no dependencies on ports at all. Otherwise I agree with the OP that you are forced to eventually upgrade everything to prevent dependency breakage and stay on the bleeding edge ports tree. Also bear in mind that compiling an upgrade for a heavily laden desktop system from ports alone can take several days.

I wrote a little about this problem myself. Mind you, I am not dissing FreeBSD but merely stating my opinion. In my own experience the ports tree is TOO bleeding edge for some people while packages are quickly outdated where they end up being at least one major version behind the upstream versions. There is space for a happy medium (a somewhat similar approach to Debian -testing for instance)

http://harishankar.org/blog/entry.php/why-i-039-ve-gone-back-to-debian

On Debian -testing I find many packages more up to date than on -STABLE for instance. I know a lot of people love ports but the gap between outdated (and quite incomplete selection) of packages on -RELEASE and even -STABLE and too-bleeding edge ports needs to be filled.

It's not an easy problem to fix, but I think manpower required for maintenance of so many branches is what is the issue here.
 
Also regarding the advise on using an "older ports tree". What about the handbook that says this?

Warning: Before installing any port, you should be sure to have an up-to-date Ports Collection and you should check http://vuxml.freebsd.org/ for security issues related to your port.

A security vulnerabilities check can be automatically done by portaudit before any new application installation. This tool can be found in the Ports Collection (ports-mgmt/portaudit). Consider running portaudit -F before installing a new port, to fetch the current vulnerabilities database. A security audit and an update of the database will be performed during the daily security system check. For more information read the portaudit(1) and periodic(8) manual pages.

Eventually things will definitely break if the ports tree is not updated.
 
As far as I see it,

If I am on an offline pc, I use RELEASE packages (because security is pretty low on my agenda when disconnected from the internet)

If I am on an online pc, I use STABLE packages which AFAIK usually have similar versions to Debian. (btw, Debian is not renowned for its up to date packages. That is one of the many reasons for ubuntu.)

If I need the latest stuff, then of course I need to compile from source. At least Ports makes it quite trouble free. The reason why I do not see the need for a "stable" ports tree is that it is unlikely you would want to be compiling old versions of applications.

Also, does the ports tree included with a RELEASE dvd, not allow you to use the dist files hosted on the freebsd ftp? Just copy them into the dist folder or set the respective environment variable to the distfiles folder on the ftp.

Although admittedly, I do also like to compile source outside of the ports tree. For example the latest mplayer, SDL, Glut because it gives me a bit more freedom.
 
@vermaden And Debian, Ubuntu, Suse, Fedora etc. pp. don't have lots of problems with broken packages, wrong configurations etc.? Sometimes you have to wait several weeks if you don't fix it yourself. I'm using Linux too, but I don't know a single package management system that has got less problems than ports. There are even some more problems,which I can easily fix whith ports. The only problem is time.

The community can easily change this problem: contributing money or contributing code and similar knowhow. I have great respect of the port commiters, it's a plethora of work they're doing.
 
oliverh said:
@vermaden And Debian, Ubuntu, Suse, Fedora etc. pp. don't have lots of problems with broken packages, wrong configurations etc.? Sometimes you have to wait several weeks if you don't fix it yourself. I'm using Linux too, but I don't know a single package management system that has got less problems than ports. There are even some more problems,which I can easily fix whith ports. The only problem is time.

The community can easily change this problem: contributing money or contributing code and similar knowhow. I have great respect of the port commiters, it's a plethora of work they're doing.

I've used Debian for more than 7 years and never had even a single issue with broken packages except when I occasionally played with "unstable" or installed third party packages without proper dependency checking. So I think you're not accurate there by clubbing several Linux distributions together. Debian's packaging policy is one of the best in the industry and worth emulating even if you happen to dislike Linux in general.

And to add to that, binary package selection in FreeBSD is nowhere near as comprehensive as in Linux.

Second issue is that I've found by personal experience that Debian -testing is more up-to-date than FreeBSD -STABLE, but I do realize that porting stuff does that time.

I think acknowledgement of certain issues will help the FreeBSD community grow. I am not saying Linux is perfect, but in the above particular aspect Linux is better than FreeBSD.

The other issue is that using packages only in FreeBSD is a bit of a pain because of lack of comprehensive package managers (pkg_add and other pkg_* tools pkg_upgrade are not good enough, I'm afraid).
 
More upstream (community) hardware, more packages
readily available. Howsoever I usually use ports.
....
Say you have 1000 ports installed but actually use
20 weekly, and 100 ports are dependencies. Then
the problem becomes downtime in those 20, while
major .so. bumps in dependencies occur.
(png, jpeg, gettext, gtk20, ...)
If you plan ahead, you can delay upgrades til you
have a few days to do it all from the console. I
use /yell/ and multiple tty's (and can use
lynx, w3m etc if X is not avail.).
....
With practice it is not so bad. (I have recently
cobbled together a few .sh to make the rebuilds
take about half as many hours as it would usually.
Unfortunately not production-ready, still beta
those .sh)
....
Furthermore, one can
"upgrade everything in March"
"ignore all upgrades 5 months"
"upgrade everything in September"
and repeat as necc.
..........
ending this particular post in case I am
not paying enough attn. to the previous
posts.
....
 
The issue with the ports IMHO is there.
But i think FreeBSD tries to do the best it can given the lack of testers, contributors, programmers, maintainers, etc...
There is no (big) money behind FreeBSD, hence all these hassles.
Acknowledging is one thing (and most people deep inside agree on that) but correcting it is another.
 
achix said:
The issue with the ports IMHO is there.
But i think FreeBSD tries to do the best it can given the lack of testers, contributors, programmers, maintainers, etc...
There is no (big) money behind FreeBSD, hence all these hassles.
Acknowledging is one thing (and most people deep inside agree on that) but correcting it is another.

Absolutely. I think you've hit it. From the time I started using FreeBSD, I realized that the kernel+userland and ports are two different entities. Kernel+userland has a strong and stable record because of the core team, while ports seem to be a little bit flakier entirely because of lack of manpower and not because of lack of ability or knowledge on the part of contributors.

Under the circumstances I think FreeBSD does a tremendous job of being on top of all the issues as it has been.

But it doesn't take away from the fact that there is a gap that needs filling. If I were less busy with my other work, I would have gladly joined as a contributor. But I frankly acknowledge that with my current level of knowledge it would take a lot of work to get there and I just haven't the time to learn the ins and outs deeper.
 
I rarely have problems with ports on my desktop PC.
For last 5 months I've been using Linux on laptop (ubuntu, sabayon), and you know what?
both of them were broken once during this time, (in the end I reinstalled ubuntu again, because it was fastest way to get laptop up and running)

during these 5 months I haven't had any serious problem with ports, that I would remember.
And I use FreeBSD ports on my Desktop, Server and it's jails
 
harishankar said:
I've used Debian for more than 7 years and never had even a single issue with broken packages except when I occasionally played with "unstable" or installed third party packages without proper dependency checking. So I think you're not accurate there by clubbing several Linux distributions together. Debian's packaging policy is one of the best in the industry and worth emulating even if you happen to dislike Linux in general.
It may be the best for you, but it is certainly not the best for everyone. Being binary based it is very fixed, and that in itself is a big problem IMHO. I've never had a debian system on which I didn't run into a case of packages being compiled with or without components I don't or do need respectively, which ultimately leads one down the path of having to manually modify source packages or compile from distribution tarballs. Both a messy headache.

And debian's idea of "stable" is misleading - in fact it really just means "old". How stable is a package with bugs that are only fixed in later versions?


harishankar said:
And to add to that, binary package selection in FreeBSD is nowhere near as comprehensive as in Linux.

The other issue is that using packages only in FreeBSD is a bit of a pain because of lack of comprehensive package managers (pkg_add and other pkg_* tools pkg_upgrade are not good enough, I'm afraid).
Agreed.

If the FreeBSD project could create ports branches in the same way there are src branches (RELENG_8, RELENG_7_0, etc.) then we might be able to get the best of both worlds. However, doing so would be of limited value if port maintainers put zero effort into maintaining the individual branches of their respective ports. Not an easy problem to solve.
 
aragon said:
It may be the best for you, but it is certainly not the best for everyone. Being binary based it is very fixed, and that in itself is a big problem IMHO. I've never had a debian system on which I didn't run into a case of packages being compiled with or without components I don't or do need respectively, which ultimately leads one down the path of having to manually modify source packages or compile from distribution tarballs. Both a messy headache.

And debian's idea of "stable" is misleading - in fact it really just means "old". How stable is a package with bugs that are only fixed in later versions?



Agreed.

If the FreeBSD project could create ports branches in the same way there are src branches (RELENG_8, RELENG_7_0, etc.) then we might be able to get the best of both worlds. However, doing so would be of limited value if port maintainers put zero effort into maintaining the individual branches of their respective ports. Not an easy problem to solve.

Stable packages also get security updates in Debian. http://www.debian.org/security/faq. That's why security.debian.org exists. ;)

And Backports are a great solution for getting more updated packages in stable.

As for "configuring" source packages, 99% of the time I don't need it. And besides, some important Debian packages tend to have different versions with different sane defaults so you can choose the package you want for your particular purpose.

E.g. Apache has different packages for "mpm-prefork" and "mpm-worker" etc.

Also the debian packaging methodology being highly modularized, the most common "source configuration" options are usually packaged as separate binary modules which fit neatly into the system like a jigsaw puzzle.

Eg. every PHP module is packaged as php-<feature> where feature is a PHP module like mysql, pdo, sqlite etc. and once it is installed it just works.
 
harishankar said:
I've used Debian for more than 7 years and never had even a single issue with broken packages except when I occasionally played with "unstable" or installed third party packages without proper dependency checking. So I think you're not accurate there by clubbing several Linux distributions together. Debian's packaging policy is one of the best in the industry and worth emulating even if you happen to dislike Linux in general.[...]

Well, I have my experience with Debian (ejabberd in Sarge for example) and once in a while I even have to use Ubuntu. It's not a Linux-Vs-BSD-thingy I'm talking of facts. I certainly will not throw Debian in the same pit as Ubuntu, but Debian has got its problems in different areas mentioned above. Finally I'm talking of testing, not stable. The latter is old as a rock and even on some servers hardly usable anymore.

http://www.suspekt.org/2010/02/27/debian-breaks-suhosin-security-feature/

Or the OpenSSL-thingy in Debian and so on. I don't not want to badmouth Debian, but it has got its problems.

>even if you happen to dislike Linux in general.

To make it short: I don't like the Debian package policy, it's better than rpm-hell but an epic fail for people adoring K.I.S.S. If I'm looking for a nice binary package management I would certainly choose pacman (Archlinux). That said, I'm using ports because they rock :)
 
@harishankar

>And Backports are a great solution for getting more updated packages in stable.

Or more bugs, problems and security breaches.

>Also the debian packaging methodology being highly modularized, the most common "source configuration" options are usually packaged as separate binary modules which fit neatly into the system like a jigsaw puzzle.

Highly modularized? Yes in terms of Debian maybe, but compared to Gentoo, ArchLinux or FreeBSD (choosing the proper options while compiling), it's rather fat.
 
> Highly modularized? Yes in terms of Debian maybe, but compared to Gentoo, ArchLinux or FreeBSD (choosing the proper options while compiling), it's rather fat.

All I know is that I don't mind a bit of "fatness" when compared to waiting several days to compile stuff on my system. Any which way I look at it, I cannot get past the bit where you compile and it takes hours for a non-trivial package and days for an entire system upgrade.

Debian in my view offers the perfect balance between convenience and flexibility which is very rare even among Linux distributions. Nothing is flawless however and I recognize your needs might not match what it offers.

> Or more bugs, problems and security breaches.

This is pure conjecture and argument for argument's sake. Well which way do you want it? You cannot have it both ways. Do you prefer that older bugs remain unfixed then? What are you trying to prove here? That FreeBSD has a better "security" policy? Because quite frankly I'm not going down that route. All my initial intention was to correct a few wrong notions of Debian -stable never getting security updates.

Finally any proof that Debian is unique in this regard? I'd say it's way better than most Linux distributions in that regard.


I'm immensely experienced, I've used a wide variety of operating systems and personally keep going back to Debian.

And with your kind permission folk, I'll stop this debate here. Obviously different strokes for different folks. :)
 
gpw928 said:
Ports are not stable. On the contrary, the ports tree is simply out of control.

...

I don't want the latest features. I do want my systems to run reliably!

May we PLEASE have a stable version of ports that is not broken almost daily?!

I happen to agree with you on this. But there are clearly political (and probably massive logistical) problems to implementing such a request: you can see from this thread, for instance, that there is not a consensus among end-users.

Whatever the outcome, I can make a suggestion to substantially reduce pain on your end. Set up a tinderbox, i.e. a separate system where you build ports for the purposes of 1) testing that they build; 2) confirming that they don't break existing services; 3) producing packages that you can deploy to your production system(s).

I've read about different strategies on this forum (from SirDice, primarily), but if you don't have a separate physical system for this purpose, then you can turn a FreeBSD jail into a tinderbox. In my case, I have a host with several jails under it. I use the host for the tinderbox, and I deploy only packages to each of the jails.
 
Back
Top