Pretty frustrated by broken ports

[donallen]

I'm a long-time Gentoo Linux user and I tried FreeBSD (7.1 Release) a few months ago, had four system crashes in 2 days due to two separate kernel bugs and gave up. But I really like the well-engineered, well-documented, professional aspects of much of the system (better along these dimensions than Linux and even Gentoo, in my opinion), so I've come back to it in a smaller way, installing it on just one of my systems, an i386 (one of the kernel bugs was amd64-specific).

I have been trying since last night to build and install the epiphany port (with portupgrade -R epiphany), because it's several minor versions ahead of the package available with the 7.1 release. I have now run into two broken ports -- x11-toolkits/py-gtk2 and security/gnome-keyring. Both fail to compile (yes, I've entered bug reports). This is a pretty egregious lack of basic software QA. If the damned things don't compile, they couldn't have been tested. What are they doing in the released ports repository?

I wish one of you who has more experience with FreeBSD than I do would tell me whether or not this is typical. If it is, it's a showstopper for me, because I really like the rolling-release capability that ports should provide but haven't for me in this case, and which I'm accustomed to after years of using Gentoo.

/Don Allen

 

[DrJ]

I have been trying since last night to build and install the epiphany port (with portupgrade -R epiphany), because it's several minor versions ahead of the package available with the 7.1 release. I have now run into two broken ports...

No, this is not typical. I've used Gnome on FreeBSD since the 2.4 days, and while there have been bugs, they usually come from the underlying Gnome code. The FreeBSD version has been pretty clean, save for a few things like not being able to pull out a mounted USB stick.

I'd bet that you have Gnome installed (from a package?) and have not updated it properly. That can cause the sorts of things you mention. Did you follow the suggestions located in /usr/ports/UPDATING? If not, you are asking for trouble.

If it really is only a minor release update, I've not seen it. And I've recently updated from 2.24 to 2.26, with a few minor updates thereafter. No major issues to speak of.

 

[gordon@]

The FreeBSD project has an automated build cluster to ensure that ports at least compile. Neither of the ports you cite are currently listed.

See http://portsmon.freebsd.org/portserrs.py

As suggested, check /usr/ports/UPDATING for details and if you continue to have issues, either check the mailing lists or email the port maintainers.

 

[donallen]

Thank you for the responses, and I'm happy to learn from both your comments that this should be atypical.

FYI, I had checked /usr/ports/UPDATING prior to attempting this and found nothing for epiphany or py-gtk2. The *is* a mention of the gnome-keyring-manager, which is not the package that failed to build in my case, but obviously related, under the discussion of upgrading gnome, which perhaps I was doing without realizing it.

In addition, after hitting the wall with portupgrade as I described, I tried portmaster overnight (because I'd seen a number of comments in various forum messages that portmaster was the best way to go), and got epiphany updated! Why this worked and portupgrade did not (nor did makes of the two ports I mentioned) is a mystery to me, though I would guess that it has to do with the ordering of the package updates. For example, I note now, from pkg_info, that py25-gtk-2.14.1 is now installed. That's a port I couldn't compile yesterday. Perhaps the compilation error was due to a header file supplied by another port that hadn't been updated and so the symbol it was complaining about was missing? gnome-keyring also got updated magically.

In any case, it looks like the comments advocating for portmaster were correct. Why portupgrade didn't work for me is still a mystery, but it could be pilot error (I did do the pkgdb preliminaries, but perhaps not correctly). In any case, I'm happy now. And thanks again for the help from both of you.

/Don Allen

 

[DutchDaemon]

If you decide to go with portmaster I advise you to deinstall portupgrade. I've seen too many examples of people using portmaster on the one hand and portupgrade utilities like pkgdb and portsdb on the other, confusing the hell out of themselves (and the pkgdb.db).

 

[DrJ]

... under the discussion of upgrading gnome, which perhaps I was doing without realizing it.

Indeed you were. Epiphany is primarily a Gnome thing, and it depends on many Gnome libraries. So when you upgrade it, you upgrade many of the various Gnome bits and pieces (since you used -R).

Why this (portmaster) worked and portupgrade did not (nor did makes of the two ports I mentioned) is a mystery to me

That happens sometimes. Usually I use portupgrade, but every so often I use portmaster because portupgrade fails. As long as you update the packages db (with pkgdb and portsdb), it works well enough.

 

[donallen]

If you decide to go with portmaster I advise you to deinstall portupgrade. I've seen too many examples of people using portmaster on the one hand and portupgrade utilities like pkgdb and portsdb on the other, confusing the hell out of themselves (and the pkgdb.db).

Can you say a bit more about this? The Handbook presents the three alternatives for updating ports, but doesn't give much of a hint about their strengths and weaknesses and how they interact with each other. I'd also be interested to know why there are three, which seems like two too many.

Having gotten my system so twisted up that it was easier to re-install than to try to fix it (I'm pretty sure I knew how, but it was going to take a lot of compilation), I'm trying to come up with an understanding and a strategy for how to get along with the ports system. I'm fairly certain that I shot myself in the foot by doing an upgrade of epiphany, upgrading all the ports on which it depends as well. After doing that, gnucash stopped working -- segfaults.

Using ldd on the gnucash executable, I found that the guile library had gotten upgraded to a newer version, and the gnucash executable was still pointing at the earlier one, the situation that Gentoo's revdep-rebuild was designed to fix. Even after rebuilding gnucash to get it linked with the newer guile library, it still failed, because I suspect that some of the ports lower in the hierarchy had stale links.

I probably could have fixed gnucash by doing a full upward recursive upgrade, but that was going to take forever (I'm doing my FreeBSD experiment on a circa 2005 Thinkpad G41 with a P4 which, despite its impressive clock frequency, is not a speed demon compared to the newer Intel processors). It was simpler to start over, which I did.

What I think I've learned is to use ports sparingly and to think carefully when you upgrade. The 7.1 release provides gnucash 2.2.6 and I would like to have 2.2.7 obtainable via the ports. I now believe the right idea is to either

cd /usr/ports/finance/gnucash
make install clean

or use one of the three port upgraders to upgrade just the gnucash port, *not* the ports on which it depends. If I should later decide to upgrade one of those, call it 'foo' for purposes of discussion, then all packages/ports that depend on foo need to be upgraded (e.g., portupgrade -r foo).

I'd be very interested in your feedback on what I've said above, or that of anyone who is a FreeBSD expert, which I am not. I'd like to know if you think it makes sense. The ports system is very nice, but it's about as far from idiot-proof as these kinds of managers get. If you are going to use ports, I've learned the hard way that you really have to think carefully about what you are doing, because you are holding more than enough rope to hang yourself.

/Don Allen

 

[DrJ]

As I mentioned above, I've had no difficulty mixing portmaster and portupgrade.

On the larger point, you have to decide if you want to use ports or packages. The latter are released less frequently than are updates to ports -- they have to be, since the binaries have to be compiled and trickle to the mirrors -- but you can get them by pointing to the right "repository."

I always have used ports, even though my main box is a seven-year-old dual Athlon. So I do understand the concerns about compilation time. Overall they have worked remarkably well as long as you don't update all the time. I usually do it every couple of months or so, unless I need a bug fix. In the interim, I just don't update the ports tree, so if I need something new, it is still in sync with what I have installed.

The main thing you should be concerned about is Gnome. That is a beast to upgrade between major versions, which I think you sort-of did (from 2.24 to 2.26). You have to be very careful to do that properly; see freebsd.org/gnome for the details. Usually you have to update *everything* you have installed that is out-of-date, and there are a few trailing updates that you have to do too. Without those, you can get your system quite messed up. They can all be fixed (ask me how I know!) but it does take some effort.

Between major Gnome updates, you are pretty safe in upgrading most anything that relies on their libraries, including the Gnome bits and pieces. It probably is the same for KDE, but I don't know it well at all.

So I set up FreeBSD by doing a minimal install, and installing xorg, Gnome and everything else from a fresh ports tree (namely, newly-installed). Then I track where Gnome is -- if one is in the minor release phase, upgrade away. When you hit a major update, you have to decide when to do the update, and give yourself a couple of days to do so. If everything is working, I wait until the .1 release unless there is something I really want. That is, I don't update the ports tree and use the old one until I am ready for the upgrade.

There are also times when one low-level library is updated, and you once again have to reinstall pretty much everything.

Some people try to keep things up-to-date much more frequently, and good for them. I have found that it takes too much time for too little benefit. OTOH, if you wait too long between updates, it is easier to reinstall than trying to do all the intermediate ones. I have one server that has a lot of custom software on it. It is running 6.1, and rather than update everything, I will reinstall one of these days.

Hope this helps some!

 

[donallen]

And perhaps just doing make install clean is not sufficient to upgrade gnucash (which I'm using as an example)? As I just found out by running
portupgrade gnucash, some of the stock 7.1 release packages are too old for gnucash 7.1 (e.g., guile), so portupgrade -r x must be done for each of them?

And to add a more concise thought to my previous post, portupgrade -R x is dangerous and should not be done gratuitously. portupgrade -r x is sometimes necessary as a side-effect of portupgrade y where y depends on x.

/Don Allen

 

[DrJ]

And perhaps just doing make install clean is not sufficient to upgrade gnucash (which I'm using as an example)?

You don't upgrade in this manner -- you only do that for newly-installed programs -- unless you use -DFORCE_PKG_REGISTER and clean up your pkgdb thereafter. This is not recommended.

As I just found out by running
portupgrade gnucash, some of the stock 7.1 release packages are too old for gnucash 7.1 (e.g., guile), so portupgrade -r x must be done for each of them?

see below.

And to add a more concise thought to my previous post, portupgrade -R x is dangerous and should not be done gratuitously. portupgrade -r x is sometimes necessary as a side-effect of portupgrade y where y depends on x.

Depends. I nearly always do a portupgrade -R when I'm upgrading a high-level program (like gnucash). But you do have to be careful, since gnucash then will dutifully upgrade everything it needs, and it is unaware of what other programs might require. That's why I update everything at one time: you take care of these sorts of issues.

So when xorg is updated, I always perform portupgrade -R xorg and it gets all the bits and pieces unless there is some new flag that you have to take care of. You can take care of that with a "make config-recursive" in the port directory to get those.

OTOH, if you upgrade something low-level, like one xorg library, and use the "-r" flag, you will reinstall pretty near everything on your computer. I doubt that's what you want to do.

So yes, do use the "-R" flag for high-level programs or metaports (but do look at UPDATING). Just be aware of where you are in the Gnome upgrade cycle, if you use it.

 

[phoenix]

Can you say a bit more about this? The Handbook presents the three alternatives for updating ports, but doesn't give much of a hint about their strengths and weaknesses and how they interact with each other. I'd also be interested to know why there are three, which seems like two too many.

Portupgrade was one of the first ports management tools. It requires the installation of the Ruby programming language runtime environment. It keeps a completely separate index database of the ports tree (the portsdb), as well as a completely separate database to track the installed apps (the pkgdb). It uses these databases to keep track of dependencies and what's installed where.

Portmaster is one of the newer ports management tools. It's a pure shell script (/bin/sh), so it doesn't require any external tools installed. It doesn't use any external databases or tools, relying completely on the ports tree framework (including INDEX) and the OS package database (/var/db/pkg/*).

There are others, but those are the two I have experience with. Back in the day, portupgrade was the king of ports management tools. However, relying on two separate, external databases that had to be kept in sync with the OS databases (and hence all the advice to constantly run "portsdb -uU" and "pkgdb -uU") makes it more of a pain than a useful tool.

Portmaster is just a shell script, and it uses all the ports/package tools that come with FreeBSD. IMO/IME, it's a nicer tool to work with, as you can easily switch between just pure ports and portmaster. The output messages are even modelled after the ports tree output.

Regardless of which ports management tool you use (including none), some good advice is to keep the ports tree updated, to always read /usr/ports/UPDATING before updating any ports, to only update ports you *need* to update (for specific features, security fixes, etc), and to only upgrade ports related to massive projects (X.org, XFce, KDE, GNOME, and the like) in one unit as part of the massive project.

 

[donallen]

Portupgrade was one of the first ports management tools. It requires the installation of the Ruby programming language runtime environment. It keeps a completely separate index database of the ports tree (the portsdb), as well as a completely separate database to track the installed apps (the pkgdb). It uses these databases to keep track of dependencies and what's installed where.

Portmaster is one of the newer ports management tools. It's a pure shell script (/bin/sh), so it doesn't require any external tools installed. It doesn't use any external databases or tools, relying completely on the ports tree framework (including INDEX) and the OS package database (/var/db/pkg/*).

There are others, but those are the two I have experience with. Back in the day, portupgrade was the king of ports management tools. However, relying on two separate, external databases that had to be kept in sync with the OS databases (and hence all the advice to constantly run "portsdb -uU" and "pkgdb -uU") makes it more of a pain than a useful tool.

Portmaster is just a shell script, and it uses all the ports/package tools that come with FreeBSD. IMO/IME, it's a nicer tool to work with, as you can easily switch between just pure ports and portmaster. The output messages are even modelled after the ports tree output.

Regardless of which ports management tool you use (including none), some good advice is to keep the ports tree updated, to always read /usr/ports/UPDATING before updating any ports, to only update ports you *need* to update (for specific features, security fixes, etc),

Yes. This is exactly what I meant when I spoke of not using portupgrade -R foo gratuitously in previous post. This is how I got myself into trouble. I unthinkingly rebuilt too much stuff, shared libraries got updated that didn't need to be, references to those libraries were left dangling, and I didn't know enough at the time to fix the problems surgically (e.g., I didn't know about libchk at the time, something the documentation could usefully mention in a broader discussion of the shared-library-update problem and how to deal with it), so ended up re-installing rather than rebuilding the world on a slow machine.

and to only upgrade ports related to massive projects (X.org, XFce, KDE, GNOME, and the like) in one unit as part of the massive project.

Yes again. I didn't attempt to deal with updating these monoliths in my initial post(s), but this is surely correct.

Thanks for the education about the pros and cons of portupgrade v. portmaster. I've now installed FreeBSD on two systems from scratch, using portupgrade where needed, and now that I'm a little older and wiser, I've managed to stay out of trouble. I'm sure I could have obtained the same results with portmaster and find your remarks about the relative cleanliness of the internals quite interesting.

Thanks again for your post.

/Don Allen

 

[donallen]

Question for you: the portmaster man page says that it will "build all ports that need updating". The question is how "need updating" is defined. If I'm upgrading port foo, and it depends on bar v1.3 or newer, and what is installed is bar v1.2, then, of course I would want bar upgraded as well. However, if bar v1.3 is installed, but there's a bar v1.4 available now, foo's requirement is met and I would NOT want portmaster to upgrade it. I suspect, though I haven't tested it, that it WILL upgrade bar to v1.4. If I'm right, any idea how to get the behavior I want, which is easy to obtain with portupgrade (just say portupgrade foo; if bar v1.3 is installed it won't complain and you're done; if bar v1.2 is installed, it does complain, aborts the upgrade, and you upgrade bar and redo the upgrade of foo). Perhaps the -G option? (The manpage says "-G prevents the recursive 'make config'", which may not be the same as preventing the recursive upgrades.)

/Don Allen

 

[phoenix]

"needs updating" means that the version available in /usr/ports/ is newer than the version listed in /var/db/pkg/

 

[kamikaze]

In addition, after hitting the wall with portupgrade as I described, I tried portmaster overnight (because I'd seen a number of comments in various forum messages that portmaster was the best way to go), and got epiphany updated! Why this worked and portupgrade did not (nor did makes of the two ports I mentioned) is a mystery to me, though I would guess that it has to do with the ordering of the package updates.

The answer is that you should have used [cmd=portupgrade]-r[/cmd] or [cmd=portupgrade]-rR[/cmd]. Unlike portupgrade that requires the -r parameter to do so, portmaster always updates dependencies.