Ports need stability

There are quite a few issues IMO. I've seen quite a few ports which compile fine but do not run, the maintainer (if any) or submiter or the last patch simply didn't test that. I've also seen people submit broken patches and then given maintainership of a port with no questions asked.
I think the concept of "Quality assurance" needs to be introduced ...
 
There's a fundamental misperception in all this.

The impression I've got after becoming a port maintainer and interacting with several committers is that the intention is already to make the portstree as stable as possible. I.e., I don't think anyone involved sees it as "bleeding edge" like CURRENT.

It's simply untrue that there is no QA effort being made.

The problem isn't the intent, it's the available resources. Manpower, machines, test harnesses. If you don't think the maintainers and committers are doing an adequate job testing, what are you doing to help? Can you offer some of your own time and machines for testing? Can you help write test suites? Can you help improve tinderbox or other testing tools?

I don't see how the proposal to introduce a STABLE portstree solves these resource problems. If anything, it makes them worse, by adding one more thing to an already-overloaded plate.
 
ckester said:
The impression I've got after becoming a port maintainer and interacting with several committers is that the intention is already to make the portstree as stable as possible. I.e., I don't think anyone involved sees it as "bleeding edge" like CURRENT.

It's simply untrue that there is no QA effort being made.

The problem isn't the intent, it's the available resources. Manpower, machines, test harnesses. If you don't think the maintainers and committers are doing an adequate job testing, what are you doing to help? Can you offer some of your own time and machines for testing? Can you help write test suites? Can you help improve tinderbox or other testing tools?

I don't see how the proposal to introduce a STABLE portstree solves these resource problems. If anything, it makes them worse, by adding one more thing to an already-overloaded plate.
Click to expand...

well said.
Until the time when coordinated massive manpower is dedicated to FreeBSD,the general concept will be as of today. There is no substitute for people.

I guess, however, that this fact, cannot be openly admitted by FreeBSD advocate activities, since it would look harmful to the very concept of promotion.

(currently writing from a firefox depending on
Code: 
libxcb.so.1 => /usr/local/lib/compat/pkg/libxcb.so.1 (0x28c20000)
:p )
 
anomie said:
Set up a tinderbox, i.e. a separate system where you build ports for the purposes of 1) testing that they build; 2) confirming that they don't break existing services; 3) producing packages that you can deploy to your production system(s).
Click to expand...

Hi,

Thanks, good suggestion. I'll do that, starting with a scratch install of 8.1.

--
Phil
 
Well, it's quite true that Ports can be unstable sometimes, but they've worked pretty well for me. Devs also send head ups before major changes.

A "stable" version of Ports would require a lot of work. A part of the infrastructure would have to be changed, so to bring this to reality, someone will be to work on this. FreeBSD is after all a volunteer project.
 
I'd also like to see a stable version of the ports tree.

I understand that lack of people and resources is a problem. But I think such a split could help reduce some of the maintainers'/committers'/developers' workload. Imagine there are two versions of the Ports tree: CURRENT and STABLE. New ports, updates to old ports and all that goes to CURRENT -- it's basically the same as it is now. Some people out in the wild -- presumably adventurous desktop users or really some people who want to devote a machine or two they own to help FreeBSD -- use the CURRENT tree. They hit problems like most of us does now and they report them and help fix them. When these problems are resolved, the now-nonproblematic ports are commited to STABLE.

This way, STABLE users would have their stable tree, and CURRENT users could easily help the project simply by testing new ports on their own machines and reporting bugs.

Of course, it would still mean that someone would have to devote their time to think out the details, do the infrastructure changes and basically get it going. But I think it would be nice to have, no?
 
Splitting the ports tree in any way will increase the workload on the ports maintainers, and it might come to a situation where patches has to be back ported between two trees. As already noted there is a QA effort, but they need resources.

By supporting the tinderbox effort either locally or the main QA tinderbox is the quickest way to increase the stability and quality of the ports tree as a whole. And if you can adopt a port that you are willing to maintain.

One thing that almost always is overlooked is that the ports tree is there as a tool to make it easier to install software, it will not do the work for you. Using ports does require you to know what you are doing with each of the softwares that you have chosen to deploy. As it is (in most cases) a compilation from source, the help you get is if they need special patches to work on freebsd.
 
manual to survive a ports disaster:

1. make a backup from all your installed ports with "pkg_info -Ea | xargs -L1 pkg_create -b" (creates packages in the current directory)
2. store /usr/ports in a ZFS dataset and make a snapshot, then "portsnap fetch update"
(alternatively you can backup them away, but this will take much more time)
3. read /usr/ports/UPDATING
4. now update your ports (for example "portmaster -a")
5. if something fails terribly:
- you can deinstall everything with "pkg_delete -a" (takes time)
- change directory to that from 1. (the backup)
- "pkg_add *.tbz" reinstalls all the packages (takes time)
- of course you could try to only deinstall/reinstall the problematic ports
- rollback the snapshot from /usr/ports
- now your system should be as before

good luck for the next time

Eric
 
eric81 said:
manual to survive a ports disaster:

1. make a backup from all your installed ports with "pkg_info -Ea | xargs -L1 pkg_create -b" (creates packages in the current directory)
2. store /usr/ports in a ZFS dataset and make a snapshot, then "portsnap fetch update"
(alternatively you can backup them away, but this will take much more time)
3. read /usr/ports/UPDATING
4. now update your ports (for example "portmaster -a")
5. if something fails terribly:
- you can deinstall everything with "pkg_delete -a" (takes time)
- change directory to that from 1. (the backup)
- "pkg_add *.tbz" reinstalls all the packages (takes time)
- of course you could try to only deinstall/reinstall the problematic ports
- rollback the snapshot from /usr/ports
- now your system should be as before

good luck for the next time

Eric
Click to expand...


Why to make backups of all installed packages, when you can simply make zfs snapshot (from your post I assume, you are using it), and if anything goes wrong, simply rollback
 
killasmurf86 said:
Why to make backups of all installed packages, when you can simply make zfs snapshot (from your post I assume, you are using it), and if anything goes wrong, simply rollback
Click to expand...

You mean /usr/local? Do you trust this procedure? Modules are stored in /boot/modules, for example from emulators/virtualbox-ose. But would be my preferred way in future.
 
If /usr/local, /usr/ports, and /var are all ZFS filesystems, then you can just snapshot all three at the same time. If things fail, just roll-back.

If you have any kernel-module ports installed, though, you will also need / on ZFS, or do a backup of / in order to restore if things fail.
 
killasmurf86 said:
Why to make backups of all installed packages, when you can simply make zfs snapshot (from your post I assume, you are using it), and if anything goes wrong, simply rollback
Click to expand...

I have everything on zfs, so I absolutely trust this, and in fact I do this quite a lot, never fails.
 
However, using -b with portmaster/portupgrade is very handy, in case you need to roll-back just a single application.
 
killasmurf86 said:
I have everything on zfs, so I absolutely trust this, and in fact I do this quite a lot, never fails.
Click to expand...

Don't forget you can snapshot UFS as well (even UFS1). It's been in FreeBSD since 5.

Code: 
gong:/h/tl# snapshot -v list /
Filesystem      Type     User   User%     Snap   Snap%  Snapshot Name   Snapshot Time
/                ufs    316MB   16.0%     42MB    2.1%  weekly.0        2010-05-09T00:00
/                ufs    316MB   16.0%      7MB    0.4%  daily.2         2010-05-13T00:00
/                ufs    316MB   16.0%      1MB    0.1%  daily.1         2010-05-14T00:00
/                ufs    316MB   16.0%      1MB    0.1%  hourly.1        2010-05-14T16:00
/                ufs    316MB   16.0%      1MB    0.1%  daily.0         2010-05-15T00:00
/                ufs    316MB   16.0%      1MB    0.1%  hourly.0        2010-05-15T16:00
/                ufs    316MB   16.0%      1MB    0.1%  oneoff.0        2010-05-15T17:27
 
But can you roll back a UFS snapshot, such that the filesystem returns to the state it was in before the snapshot was taken?

I though UFS snapshots were more geared toward backups, where you would have a never-changing source tree to read from even if the backups take hours.

Similar to how LVM snapshots work on Linux (although, those snapshots are completely pointless as you have to reserve space in your volume group for the snapshots, and you have to specify a size for the snapshot, and and and and).
 
Oxyd said:
I'd also like to see a stable version of the ports tree.

I understand that lack of people and resources is a problem. But I think such a split could help reduce some of the maintainers'/committers'/developers' workload. Imagine there are two versions of the Ports tree: CURRENT and STABLE. New ports, updates to old ports and all that goes to CURRENT -- it's basically the same as it is now. Some people out in the wild -- presumably adventurous desktop users or really some people who want to devote a machine or two they own to help FreeBSD -- use the CURRENT tree. They hit problems like most of us does now and they report them and help fix them. When these problems are resolved, the now-nonproblematic ports are commited to STABLE.

This way, STABLE users would have their stable tree, and CURRENT users could easily help the project simply by testing new ports on their own machines and reporting bugs.

Of course, it would still mean that someone would have to devote their time to think out the details, do the infrastructure changes and basically get it going. But I think it would be nice to have, no?
Click to expand...

As you say package goes into CURRENT, people test (or just use) it, report problems, problems get fixed, no new bugs reported (for a while), package moved to STABLE . . . then new problems are reported (can happen, particularly with libs and also more people using stable than current). The only result is the already hardworking port owners now have to maintain 2 versions: a CURRENT, and a [so called] STABLE version.

I'm one of those that has little trouble with the current ports system and don't see the need to change it. If I need a port (or package) I install it, and if it works fine I keep using it as-is. Unless there is some new feature in a later release that I absolutely must have why upgrade? Sure when installing some new package I've had to add the occasional symlink (i.e. quite common link libjpeg.7 --> libjpeg.8), but so what? Everything I use is still working.
 
harishankar said:
As for "configuring" source packages, 99% of the time I don't need it. And besides, some important Debian packages tend to have different versions with different sane defaults so you can choose the package you want for your particular purpose.

E.g. Apache has different packages for "mpm-prefork" and "mpm-worker" etc.
Click to expand...

I agree.

One thing that I believe will lighten the load on port maintainers is reducing the number of knobs (either "make config" or WITH_*). Doesn't a port maintainer have to at least confirm that each knob combination is buildable? And I just can't imagine how a dependency tree can be maintained when flipping a knob can change large parts of it. We already have a few ports that do this (i.e. vim vs. vim-lite); I don't see why not.

I appreciate the hard work porters are doing (esp. the Xorg), but I do feel our ports system is more labor intensive than others'.
 
>Doesn't a port maintainer have to at least confirm that each knob combination is buildable? And I just can't imagine how a dependency tree can be maintained when flipping a knob can change large parts of it.

@darehanl, no. The default knobs are the recommendation. If you play with it, then you should know what you're doing. Finally FreeBSD isn't Ubuntu or Fedora, it's more like Gentoo or Slackware, it's an operating system for the seasoned user.

>I appreciate the hard work porters are doing (esp. the Xorg), but I do feel our ports system is more labor intensive than others'.

Did you ever build rpm or deb packages? Did you ever experience cyclic dependencies in Debian? Well, there are usually more problems with binary packages then sourcecode.

>One thing that I believe will lighten the load on port maintainers is reducing the number of knobs

This is one of the huge advantages of using ports. If you want to reduce the load, test ports, file problem reports, file bug fixes, file updates etc. pp. It's that easy, anyway it's of almost no use to lament in some forum.
 
How can a messed up port hurt so bad?

did you do it

- on a live production server
- at peek times
- without proper backup measures ?


I think this thread has no real direction, and doesn't add much to the rest of the awful this-versus-that-OS debate.
 
I hate to open up an old thread, but I just read this and would like to add to the conversation in hopes of learning something. Basically I experienced a rather similar issue just a few weeks ago. First off, I always have an up-to-date ports tree, and regularly run portaudit -Fa. Well as some of you know, there is a vulnerability out there for kdebase-workspace-4.3.5 (I hope thats right as I am at work right now) Anyway, so I did a portupgrade on that which wanted to upgrade to 4.4.1. All is good and dandy until it decides it needs a new version of jpeg (was using jpeg-7, now it requires 8) and also an upgrade of png. As said previously that just about means I might as well install FreeBSD 8.1 Beta since just about every port and/or package needs to be updated. Eventually it got so bad that I couldnt update one without having a newer version of a port that required the old dependency. SO it was pretty much like a deadlock. After like 5 hours of being angry and trying to resolve the issue I ended up reformatting, and refusing to upgrade any ports. So now I have a vulnerability with kdebase-workspace. Having wasted an entire night and not accomplishing anything other than a headache I beg the question: How do you prevent this, and how do you know whether or not this will happen before you get in over your head? Please advise.
 
z662 said:
All is good and dandy until it decides it needs a new version of jpeg (was using jpeg-7, now it requires 8) and also an upgrade of png.
Click to expand...

Solving that case is generally pretty easy (it works at least for png/jpeg/curl/libogg ports on my box).

Code: 
# cd /usr/ports/category/someport
# make deinstall package clean
# make deinstall
# pkg_add -r someport
# pkg_add -f /usr/ports/packages/All/someport-X.Y.Z.tbz

You will end up with BOTH versions of the libraries for that port (jpeg/png/libogg/...), but of course some files overlap:
 
Code: 
% pkg_info -L png\*   
Information for png-1.2.40:

Files:
/usr/local/man/man3/libpng.3.gz
/usr/local/man/man3/libpngpf.3.gz
/usr/local/man/man5/png.5.gz
/usr/local/bin/libpng-config
/usr/local/include/libpng/png.h
/usr/local/include/libpng/pngconf.h
/usr/local/lib/libpng.a
/usr/local/lib/libpng.so
/usr/local/lib/libpng.so.5
/usr/local/libdata/pkgconfig/libpng12.pc

Information for png-1.4.1_1:

Files:
/usr/local/man/man3/libpng.3.gz
/usr/local/man/man3/libpngpf.3.gz
/usr/local/man/man5/png.5.gz
/usr/local/bin/libpng-config
/usr/local/include/libpng/png.h
/usr/local/include/libpng/pngconf.h
/usr/local/include/libpng/pngpriv.h
/usr/local/lib/libpng.a
/usr/local/lib/libpng.so
/usr/local/lib/libpng.so.6
/usr/local/libdata/pkgconfig/libpng14.pc
 
These that have '2' on the beginning of the line are overwritten:
Code: 
% pkg_info -L png\* | sort -n | uniq -c | sort -r
   4 
   2 Files:
   2 /usr/local/man/man5/png.5.gz
   2 /usr/local/man/man3/libpngpf.3.gz
   2 /usr/local/man/man3/libpng.3.gz
   2 /usr/local/lib/libpng.so
   2 /usr/local/lib/libpng.a
   2 /usr/local/include/libpng/pngconf.h
   2 /usr/local/include/libpng/png.h
   2 /usr/local/bin/libpng-config
   1 Information for png-1.4.1_1:
   1 Information for png-1.2.40:
   1 /usr/local/libdata/pkgconfig/libpng14.pc
   1 /usr/local/libdata/pkgconfig/libpng12.pc
   1 /usr/local/lib/libpng.so.6
   1 /usr/local/lib/libpng.so.5
   1 /usr/local/include/libpng/pngpriv.h
 
By this way, You have OLD version to have OLD ports/packages installed working correctly and a NEW version and NEW sources/headers under /(...)/include to satisfy new ports.

I am using this method for some time and all things work great.

The problem comes if a dependency for a lot of ports have a vulnerability, then You will have rebuild it all to be secure.

TO ADMINS: I needed to split my post to small parts because I got these errors:
Code: 
Error 503 Service Unavailable

Service Unavailable
Guru Meditation:

XID: 400838167
Varnish
 
You must log in or register to reply here.
Back
Top