• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Ports need stability

z662

Well-Known Member

Thanks: 3
Messages: 292

#51
I see vermaden, so essentially the only 'good' way to resolve the issue involves the installation of packages? If thats the case, what is the benefit of installing ports in the first place? I always was under the impression ports where preferred since you can customize the application however you wanted, and the fact that it is being compiled on your machine hence being the 'optimal' installation method. (I do not really know the reasons, but I am guessing when your machine compiles it, it is better???)
 

vermaden

Son of Beastie

Thanks: 902
Messages: 2,578

#52
z662 said:
I see vermaden, so essentially the only 'good' way to resolve the issue involves the installation of packages?
Port after being built becomes a installed package generally, You can also create a *.tbz package in the process, it does not matter how the port/package is installed, by make install compilation process or only by adding an already compiled *.tbz packages, its the same.

If thats the case, what is the benefit of installing ports in the first place?
I do not use port if I do not must, but there are no pacakges for ALL ports (lame for example).

I always was under the impression ports where preferred since you can customize the application however you wanted
None of the methods is prefered, You use what You like most or what better works for You.

and the fact that it is being compiled on your machine hence being the 'optimal' installation method.
Its non important, You only loose time.
 

jb_fvwm2

Daemon

Thanks: 171
Messages: 1,672

#53
I should update my post (#16 in this thread) to
include the /usr/local/lib/compat/[place old .so. here]
method I've since tried (see the other thread that
mentions it). That at least lends "stability" to
some of the upgrades (here anyway).
 

oliverh

Aspiring Daemon

Thanks: 38
Messages: 557

#54
>Its non important, You only loose time.

That's maybe true for you, but as general point of view it's mere nonsense. We aren't talking of Gentooish madness (I don't count numbers), but customizing. Customizing is essential for many people, I don't need lots of rubbish installed per default (see e.g. Debian). I prefer slim applications with few dependencies. Time is of course an argument, but packages aren't the answer.

Furthermore packages in FreeBSD are a pain in the backside. Over a period of time you will _always_ experience lots of problems, especially if you have to use certain ports. Apart from that, handling of packages is a major pain in the backside too. It's the sorry-state of todays FreeBSD.

Do you know Greg Lehey? Well ...

This is another nail in the coffin of FreeBSD. What new user is going to install it if he can't even work out how to start X? On the same hardware, Ubuntu installs and works out of the box. There's a certain “submit patches” mentality in the FreeBSD project, and it's lethal. Ubuntu is nice and comfortable for people who want what it provides; once you get outside that area, it becomes more difficult. Somehow that's preferable to gratuitous changes that mean that even people like myself with decades of experience can't get the bloody thing to start.
Sunday, 6 June 2010, http://www.lemis.com/grog/diary-jun2010.php#bottom

or

Building a new FreeBSD machine is becoming more and more of an issue. I had planned and started an upgrade anyway last month, but given it up because of some weird bug that seems to bite only me. And though building ports from source seems like a good idea, and in the past I've supported it, it's becoming more and more fragile in the course of time: I would bet that I wouldn't be able to install all the software I wanted to without some configuration or compilation issues. And somehow, though processors are now nearly 1000 times faster than they were when I started using FreeBSD, building software takes about the same time.
or

Somehow it's like the end of an era: using Linux instead of BSD simply because it's easier. Of course, it's not as simple as that, but it seems to be the first step. Sad, somehow. In my experience, BSD has always been cleaner and more reliable, but it's getting a bit rough at the edges.
both from Saturday, 15 May 2010, http://www.lemis.com/grog/diary-may2010.php#bottom


I startet with FreeBSD 5.0 because of Leheys articles and musings in his diary. I don't leave FreeBSD, but I have to use Linux once in a while for more ambitious work. And it's "nice" to see, that even someone, as essential to FreeBSD, like Greg Lehey made similar experiences. That said, I'm aware of the many problems especially regarding ports and the stepchild packages, but at the end of the day you need something which is able to do more than email and browser.

I'm sorry it's rather offtopic, but I'm fed up with this kind of hype aka "there is no problem, Linux has got lots of problems too".
 

joag

Member

Thanks: 1
Messages: 39

#56
The way I use to install upgrade ports

I don't know whether this is going to help or not as this post is too long and I think it'll continue to grow.

I'm a big Linux fan and I've installed almost any distro you can think of just for the sake of knowing which one was the right one for me, at the end Debian won because it is easy to maintain, use and I use it as my web server; for the desktop side I decided to use Ubuntu as it is Debian based but I got tired of instability brought by to frequent changes in new releases; aptitude is the best package manager I've ever used IMHO though.

Going back to what should be in this post and replies
For installing software I prefer ports
portsnap fetch extract ==> when you first install the ports system
portsnap fetch update ==> used when you want to update the ports tree

The last option you run it every few weeks or whenever you want and forget about running it if your current packages are not up2date from your current ports tree because you can cause or not sever issues to your system depending on the update(just don't do it til all our packages are up2date).

How do yo know what to update:
[CMD="+"]portversion -l "<" [/CMD]

This last command give you the packages needing an update according to the current installed ports tree.

how to upgrade this packages:
portupgrade package-1 package-2 ... package-n --batch

--batch means run the update in batch mode :D which for short is say yes to everything so you don't have to provide any input it just everything by defaults.

And all this combined with some experiences dealing with broken ports which you'll get after few months working on a BSD based system; I'm relatively new dealing with this things and this is what I've been using without problems and now FreeBSD is my main desktop OS.

Hope this can help a bit to newcomers to FreeBSD.
 

darehanl

New Member


Messages: 6

#57
joag said:
I don't know whether this is going to help or not as this post is too long and I think it'll continue to grow.

I'm a big Linux fan and I've installed almost any distro you can think of just for the sake of knowing which one was the right one for me, at the end Debian won because it is easy to maintain, use and I use it as my web server; for the desktop side I decided to use Ubuntu as it is Debian based but I got tired of instability brought by to frequent changes in new releases; aptitude is the best package manager I've ever used IMHO though.
OT: Pacman is my preferred package manager:)
joag said:
How do yo know what to update:
[CMD="+"]portversion -l "<" [/CMD]

This last command give you the packages needing an update according to the current installed ports tree.
Another alternative would be:
[CMD=""]pkg_version -vI | grep -v '='[/CMD]
The 'I' means check against /usr/ports/INDEX-*, so if you have any local changes to the ports tree take it out.

joag said:
how to upgrade this packages:
portupgrade package-1 package-2 ... package-n --batch
portmaster -Db package-1 package-2...
 

wblock@

Administrator
Staff member
Administrator
Moderator
Developer

Thanks: 3,558
Messages: 13,856

#58
joag said:
How do yo know what to update:
[CMD="+"]portversion -l "<" [/CMD]

This last command give you the packages needing an update according to the current installed ports tree.
I prefer
# portversion -vL=
because it doesn't have any shell redirection symbols. :)

how to upgrade this packages:
portupgrade package-1 package-2 ... package-n --batch
The -r option is useful, and I use it by default:
# portupgrade -r port1 port2
will upgrade port1, port2, and anything that depends on either. This helps keep things from breaking.

-C shows you all the config screens and lets you set options. For ports that you haven't done make config on before, at least. It shows all the config screens up front, before any building, so you can set them all and then leave it to build and install everything.
 

cracauer@

Member
Developer

Thanks: 2
Messages: 65

#59
I have posted this in various forums, the most complete version on slashdot.

I am convinced that we need to have a stable branch of the ports tree analogous to what Debian does.

I know everybody has different pet peeves, mine is Xorg. Xorg breaks things for me at a very rapid pace starting from 7.2. It is unbearable. That is bad enough as such, and it gets worse by the fact that this affects both my Linux and my FreeBSD boxes. But the real trick is that my supposedly most stable OS - FreeBSD - upgrades me to broken new Xorg versions first, not last like it should. Other people have other problems but alas the situation on Debian is better since Xorg is in a stable third-party software branch on Debian.

Now, I won't slam FreeBSD too much since Debian/testing now also has unusable and Debian/stable too broken Xorg. I have switched the first Desktop to run Xorg (an older version) in a chroot tree and expect to run like this for years to come. So FreeBSD isn't too much worse off anymore.

I have a more complete rant here:
http://bsd.slashdot.org/comments.pl?sid=1396293&cid=29675375
 

fwaggle

Member

Thanks: 6
Messages: 62

#60
cracauer@ said:
I am convinced that we need to have a stable branch of the ports tree analogous to what Debian does.
To play devil's advocate, I don't really see how this would make things better. It's my understanding most of the "omfg breakage" with ports is discovered when people try to use it - a certain amount is avoided through testing, I'm sure. But I don't really see how having a slower-moving ports tree will make things any better. The problems that are plaguing the current system right now don't surface until mass amounts of people try to use them, so the issues will still make it into the "stable" ports tree anyway before being caught.

The only thing I can see that might help matters is if you had the option to keep ports in sync with the versions they were at the time your release came out (-STABLE and -CURRENT would obviously track the current ports tree), then have some retention on old tarballs and backport security issues, and doing both until the EOL for a given release. Except then you're basically asking each and every port maintainer to exponentially increase their workload - if just a small percentage of them don't want to do it, you're left with what you have now... either track the ports tree as it moves and hope stuff don't break, or sit at your -RELEASE's ports tree and hope the tarballs stay put and there's no security issues.

I think a "-RELEASE" ports tree (I think calling it "-STABLE" is a misnomer, the tree we have right now fulfills the obligations of what FreeBSD calls "-STABLE") would just increase the workload and you'd still have almost the same amount of breakage you have now. What you're basically doing is insisting on other people doing the testing and getting tripped up by the breakage - if no one wants to do that, who's going to do it?
 

gilinko

Well-Known Member

Thanks: 57
Messages: 416

#61
cracauer@ said:
Now, I won't slam FreeBSD too much since Debian/testing now also has unusable and Debian/stable too broken Xorg. I have switched the first Desktop to run Xorg (an older version) in a chroot tree and expect to run like this for years to come. So FreeBSD isn't too much worse off anymore.
The main "we need a stable ports system" comes from using FreeBSD as a desktop and thusly depend on Xorg, which for the last two years has been in a transition state starting with Dbus, hald, xrandr etc. A _lot_ of new and unstable technologies has been incorporated and has broken a lot of things over and over and over again.

I have been using both Fedora and Ubuntu on the desktop and not one release has gone by where it hasn't been broken someway on the desktop and needed either an experimental patch not worse there is no solution. For Fedora 13 is was the total absence of drivers for ANY ATI/AMD GFX card due to a bump in the Xorg version.

So it's not the ports version of Xorg that's unstable, it's the Xorg software itself... Which is one reason I use Mac OS X for my desktop now.

We rely on the ports tree use the latest released version of a software. Not a RC, Beta or something else, and rely on the testing done by the ones that do the releases of their software(like Xorg, Mozilla, Apache etc). The main problem is not the individual softwares but the integration of them all. Especially when we are talking about the desktop (ie Xorg/Gnome/KDE etc), and frankly that's not a problem for the ports to handle but something that has to be solved upstream.

In an ideal world of course... ;)
 

bsd10

Member

Thanks: 3
Messages: 72

#62
RELEASE Ports Tree

I think we should have a ports tree that only has security patches for the ports in the most recent STABLE branch extended release. A full FreeBSD distribution with binary security updates for base and packages could be helpful to a lot of users.

If anyone is interested in doing this, I would be willing to contribute a server to host the ports repository and build the packages. It would probably make sense to start with the ports in 8.1-RELEASE when it comes out.

Edit: The following was added after my post was merged here

cracauer@ said:
I am convinced that we need to have a stable branch of the ports tree analogous to what Debian does.
I completely agree.

fwaggle said:
I don't really see how having a slower-moving ports tree will make things any better.
What I would propose is not a slower-moving tree, but rather a RELEASE tree. It would work just like the RELEASE branch, by providing only security updates, so breakage would be very unlikely, as no new features would be added unless absolutely necessary. This would allow users to do a full binary security update of both the base system and ports. Each RELEASE comes with a fully functional ports tree, but it is never updated. If security updates were provided for that tree, I think it could potentially be the best option for all users like me who love FreeBSD for its stability.
 

chalbersma

Member

Thanks: 3
Messages: 92

#63
ZFS on a White Horse?

From my understanding ZFS could significantly help with this process (I currently don't use ZFS but I will when I get my next system :).

You could install your system, run it as normal. Then with zfs create a snapshot of the system. Now upgrade your ports. You new ports break everything? Rollback to your snapshot.

If someone with ZFS could give me a heads up on whether this will work or not that would be cool.
 

cracauer@

Member
Developer

Thanks: 2
Messages: 65

#64
fwaggle said:
To play devil's advocate, I don't really see how this would make things better. It's my understanding most of the "omfg breakage" with ports is discovered when people try to use it - a certain amount is avoided through testing, I'm sure. But I don't really see how having a slower-moving ports tree will make things any better. The problems that are plaguing the current system right now don't surface until mass amounts of people try to use them, so the issues will still make it into the "stable" ports tree anyway before being caught.
We are talking different things here. You talk ports breakage as in the ports tree doesn't build, has dependency problems and the like.

I talk about a perfectly fine building ports tree pulling in software packages that are broken upstream.

fwaggle said:
The only thing I can see that might help matters is if you had the option to keep ports in sync with the versions they were at the time your release came out (-STABLE and -CURRENT would obviously track the current ports tree), then have some retention on old tarballs and backport security issues, and doing both until the EOL for a given release. Except then you're basically asking each and every port maintainer to exponentially increase their workload - if just a small percentage of them don't want to do it, you're left with what you have now... either track the ports tree as it moves and hope stuff don't break, or sit at your -RELEASE's ports tree and hope the tarballs stay put and there's no security issues.

I think a "-RELEASE" ports tree (I think calling it "-STABLE" is a misnomer, the tree we have right now fulfills the obligations of what FreeBSD calls "-STABLE") would just increase the workload and you'd still have almost the same amount of breakage you have now. What you're basically doing is insisting on other people doing the testing and getting tripped up by the breakage - if no one wants to do that, who's going to do it?
Right, that won't work.

The Debian model is the only model I can see working.
 

cracauer@

Member
Developer

Thanks: 2
Messages: 65

#65
gilinko said:
The main "we need a stable ports system" comes from using FreeBSD as a desktop and thusly depend on Xorg, which for the last two years has been in a transition state starting with Dbus, hald, xrandr etc. A _lot_ of new and unstable technologies has been incorporated and has broken a lot of things over and over and over again.
My impression is more than the Xorg people just make everything run that a user coming from Windows might expect and although they leave options in to switch to things that traditional Unix did better those options become broken. This is both a developers problem and a QA problem. Neither seem to be interested in "powerusers". It's all very interactive and all developed without much of a background in what's out there in the Unix world. I mean they just broke xterm by just not knowing that certain modifier keys with certain chars are in use by some curses apps.

gilinko said:
We rely on the ports tree use the latest released version of a software. Not a RC, Beta or something else, and rely on the testing done by the ones that do the releases of their software(like Xorg, Mozilla, Apache etc).
In the Xorg case the problem is upstream, and it is a perfectly valid but broken release.

gilinko said:
The main problem is not the individual softwares but the integration of them all.
Not for me. The main problem is broken upstream releases. I can deal with integration problems on my own (in fact the ports tree usually takes a week to go through cleanly on my Thinkpad).

gilinko said:
Especially when we are talking about the desktop (ie Xorg/Gnome/KDE etc), and frankly that's not a problem for the ports to handle but something that has to be solved upstream.

In an ideal world of course... ;)
If I didn't lack any interest in X11 programming I'd split Xorg at the 7.1 or 7.2 release point right now.

But that's the problem, right? Those people who just want a working Xorg for traditional Unix functionality do not want to hack X11. So they get taken on a wild ride.

Add to that their extremely messy build system with everything splattered into different source tar files, but requiring their own same specific versions in a strict dependency tree, their "solution" to that problem that is a Python script that didn't make it through 1/10th of the tree (didn't keep error messages, sorry) and the inability to compile into anything but default paths (yes I just tried, no go, too many -Is and -Ls missing for the new prefix and a couple hardcoded /usr/.... places).

Apple of course is a source of sanity against some of that klickibunti Linuxism but they have opted not to use X11 in the first place.
 

cracauer@

Member
Developer

Thanks: 2
Messages: 65

#66
chalbersma said:
From my understanding ZFS could significantly help with this process (I currently don't use ZFS but I will when I get my next system :).

You could install your system, run it as normal. Then with zfs create a snapshot of the system. Now upgrade your ports. You new ports break everything? Rollback to your snapshot.

If someone with ZFS could give me a heads up on whether this will work or not that would be cool.
I can rollback by taring up a safety copy of /usr/local and /var/db/ports just fine, I don't need ZFS.

But I need updates for software that didn't opt to break things in new releases. Security fixes just for starters. You can't just not update.

And Xorg breakage does not get repaired. You can't rollback your ports tree until it's fixes. It's broken in releases upstream. It's not a one-liner fix in your port's Makefile.
 

fwaggle

Member

Thanks: 6
Messages: 62

#67
cracauer@ said:
We are talking different things here. You talk ports breakage as in the ports tree doesn't build, has dependency problems and the like.

I talk about a perfectly fine building ports tree pulling in software packages that are broken upstream.
No, I think we're talking about the same thing. For example, Xorg being broken upstream - in my very, very limited understanding of the problems, I get the impression that most of the problems won't be found until the masses start using the given port in a bunch of different configurations. If there was any kind of test case, I'd be willing to bet most of the port maintainers wouldn't knowingly put a broken port into the tree in favor of an older one that works without a good reason. I'm sure the person who's email address is in the port doesn't want to be inundated with angry or exasperated emails without a good reason behind it.

So I guess what I mean is that having one "-STABLE" (and calling the current ports tree "-CURRENT") doesn't really work because where do you draw the line at something "not working"? If you have a ports system that allows you to keep the versions you have and just add security patches (a la tracking RELENG_X_Y) then the workload for ports maintainers goes up massively, further increasing the likelihood something will break.

Finally, if I'm not mistaken, ports pulls many of it's tarballs in from the third party sites - how can you guarantee those tarballs will be retained until the EOL for a given release (which is probably a pretty reasonable support time frame anyway)?

I guess what I'm trying to say is the root cause of the problem appears to be lack of manpower in the ports tree in general, and I haven't seen a single proposed solution that wouldn't exacerbate that root cause. I haven't seen anyone willing to volunteer to maintain a tree, and I think nominating every port maintainer to at least double their workload is probably not a good move. If you want to make sure your ports tree doesn't become broken, keep your ports tree static and then backport security patches yourself. :D
 

chalbersma

Member

Thanks: 3
Messages: 92

#68
fwaggle said:
...
Finally, if I'm not mistaken, ports pulls many of it's tarballs in from the third party sites - how can you guarantee those tarballs will be retained until the EOL for a given release (which is probably a pretty reasonable support time frame anyway)?
...
If I'm not mistaken the checksums in the port assure this.
 

mickey

Well-Known Member

Thanks: 52
Messages: 359

#71
It is my opinion, that having a STABLE branch of the ports tree would not improve the situation, but only increase the workload and steal resources that would be of more benefit, if spent otherwise. Having a certain amount of QA in the ports tree should not be an option, but the default.

I have been using the ports tree exclusively like forever now. The thing that has bothered me the most within the last 1-2 years now, has been that updates which cause lots of dependant ports needing to be upgraded too, have not been deployed in a coordinated fashion. We have seen updates to jpeg, png, gettext, libtool and whatnot. Nearly each of these required you to rebuild something like 75% of all your installed ports, causing major headaches. Everytime you were thinking: "hey, i have my system up to date again", the next major breakage was just around the corner.

Also looking at some ports, I get the impression, that someone has to apply the same patches with every new version of the software that gets released. And I keep asking myself, why don't these patches go upstream, so that the effort of porting the next release of the software will eventually become nil. Do the upstream developers simply not care about the 'portability' of their software? I understand, that many software gets mainly developed on linux, but does that imply to be ignorant to the fact, that people using other OSes would like to use that software, too?
 

jb_fvwm2

Daemon

Thanks: 171
Messages: 1,672

#72
mickey said:
The thing that has bothered me the most within the last 1-2 years now, has been that updates which cause lots of dependant ports needing to be upgraded too, have not been deployed in a coordinated fashion. We have seen updates to jpeg, png, gettext, libtool and whatnot. Nearly each of these required you to rebuild something like 75% of all your installed ports, causing major headaches. Everytime you were thinking: "hey, i have my system up to date again", the next major breakage was just around the corner.
edit
the below does not apply to "rebuild everything" with portmaster or portupgrade.
I have too many ports installed to ever do that...
also, I am not sure about the use of /usr/local/lib/compat/pkg as opposed to /usr/local/
lib/compat/...
(using usually tty's concurrently, sometimes with "portmaster -d port port port "
END edit


I thought so too for years. Then a few weeks ago a thread here
pointed out that breakages could be lessened by interim copying
of the .so. files to be obsoleted temporarily to /usr/local/lib/compat.
...
Just using it with gettext so far (2 .so.) but anticipate using it for
curl, png, jpeg, (etc) as well. (Having updated a subset of
the gettext-dependent ports but not others). For me, turns the problem
of port breakage wholly around for the better. Only caveat, one must
remember to eventually remove the stale .so.'s (maybe during a buildworld
cycle).
....
I was thinking, maybe a /port/ or a switch to pkg_delete could place
.so. 's in /usr/local/lib/compat and setup an "at" command to
every-so-often echo to the screen: "have you removed the 'something'
stale .so. files yet" or something. (and bail out maybe if there
are too many .so. files to put there, or if the /port/ is too
intermediate (has too many dependencies) and might not work as
smoothly. (I suspect there would be untoward effects, but have
not tried any other than /gettext/).
 

mickey

Well-Known Member

Thanks: 52
Messages: 359

#73
jb_fvwm2 said:
I thought so too for years. Then a few weeks ago a thread here
pointed out that breakages could be lessened by interim copying
of the .so. files to be obsoleted temporarily to /usr/local/lib/compat.
Normally portupgrade per default seems to copy outdated shared libs to /usr/local/lib/compat/pkg. So that is not the point. For me the 'breakage' is that I cannot use my system for some 1-2 days, cause it is busily recompiling half of the installed packages and me having to use my old notebook in the meantime :e

jb_fvwm2 said:
Only caveat, one must remember to eventually remove the stale .so.'s (maybe during a buildworld cycle).
Just through with that yesterday, after updating gettext and stuff. Deleted all stuff from /usr/local/lib/compat/pkg, located all binaries/libraries with unresolved dependencies, and reinstalled those. Now everything seems to be working smoothly again ... until the next major update :p
jb_fvwm2 said:
I was thinking, maybe a /port/ or a switch to pkg_delete could place .so. 's in /usr/local/lib/compat and setup an "at" command to every-so-often echo to the screen: "have you removed the 'something' stale .so. files yet" or something.
How about the Windows approach?
Code:
System indicates that the shared object foobar.so.42
is no longer used by other system components.
Do you wish to delete it permanently?
;-)
 

cracauer@

Member
Developer

Thanks: 2
Messages: 65

#74
DutchDaemon said:
Checksums do not guarantee that the author himself will not remove tarballs for older versions of his software ...
Not an issue.

Third-party site hosted tarballs are always copied to freebsd.org sites that serves as backup for as long as the port references that tarball. Even if the original disappears the freebsd copy will be there.
 

Alt

Aspiring Daemon

Thanks: 82
Messages: 726

#75
Dunno when ports changed perl5.8 to 5.10, but now im updating all pkgs with portmaster.
Got loads of errors thats horror :p
Most of errors is that pkg_info says perl module is installed. But they are in wrong(old) directory so perl dont see them..

I wonder if there a way to just apply security patches for installed pkgs (as freebsd-update), so i can run server non-stop