Ports Tree Philosophy Upgrade Needed

fossette

fossette

The computer world is becoming more complex every day, and resources to take care of FreeBSD are finite. With that combination, the grab of Murphey's Law is just a release away… And Oko's quote "most FreeBSD developers use MAC" doesn't help either.

The short version:
The FreeBSD Ports Tree should be able to handle older port releases for improved desktop stability.

The solution inspiration:
I had some issues with www/firefox, so I decided to upgrade it. I upgraded it once before, so what's the big deal, right? Well, this time, www/firefox requires a new version of multimedia/ffmpeg which fails to install due to missing doc files. Wtf? This multimedia/ffmpeg port mess also affect ports that require multimedia/ffmpeg like multimedia/vlc. How useful is my FreeBSD desktop now?

Before committing to a new port release, the port must be thoroughly tested. Perhaps that was easy in the good'ol days, but it's now wishful thinking. Systems are infinitely more complex, and they have a huge variety of peripherals. So many things to test, so little time. Bugs/issues will eventually slip through, and the end-user ends up resuming the port's test phase unknowingly.

Here's a solution to minimize breaking users' FreeBSD systems. Let older versions of a port coexist in the Ports Tree until no other port requires them. For example, you want to upgrade multimedia/ffmpeg to 2.8? Fine! Go for it! But keep 2.7 as well until multimedia/vlc, www/firefox et al., have been expressively tested using the newest 2.8 version. So, if multimedia/ffmpeg 2.8 has issues, other ports that are still based on the multimedia/ffmeg 2.7 will still be able to run. THAT'S DESIRED STABILITY FOR USERS.

In order to have several versions of the same library in the same space, the version number must be part of the file name or the file path, just like tarballs are named. Obviously, Makefile files will have to link to the specific library version that the port has been thoroughly tested with. Besides, some ports of the Ports Tree are already version specific. Why not extend this feature to all libraries? That being said, there's no need to change the entire Ports Tree to implement this solution right away. Ports can gradually be adjusted when a new version is ready to be committed in the Ports Tree. VOILA!

Needless to say that the maintainers' sanity will be spared with this solution. Possible damage will be limited to one port, and the few users that have decided to upgrade in the meantime. That's less people affected by a bad port release, less stress for the maintainers, and 'relatively' more time for them to address the issue (because everybody got offline lives). THAT'S A WIN-WIN SITUATION.

Looking forward for your input and suggestions! Hopefully, the FreeBSD Developers will find this solution appropriate.

Dominique.
 
fossette, thanks a lot for putting attention on this problem.

I'm a user of FreeBSD desktop (PCBSD) since 9.1 version. A have the problem of software stability during all this time. When one port or package needs to be updated, I must update almost ALL ports/packages, and very often this update repairs a problem of one software but introduces another problem in other software. The last example - I needed to update Firefox, I finished by updating all packages and now my LibreOffice does not work anymore - I need to update my OS from 10.1 to 10.2 to get it working... But there are some other software I'm using which does not work on 10.2!! So updating my OS I will have other stuff not working anymore. And so on... So the problem exists, it concerns (mainly) desktop users, and a solution is really needed.

As about possible solutions - I think that coexistence of several ports in the ports tree is a possible solution, but definitely NOT the best one. Another solution should be used, based on isolation of software.

At the time of PCBSD 9.1 it was using PBI system, where every software was coming with all necessary dependencies in one package. The packages were installed in separate folders, and symlinks were created in main tree for libs. Sure, such system was buggy and difficult to maintain (for example, any external plugin must be integrated at the moment of PBI creation). But globally, PBI system was able to solve the problem of coexistence of several versions of software, I could delete the actual version of software and reinstall the previous one. And it was working! Finally, PCBSD guys changed the PBI system and finished by using the 'classic' packages approach. So the initial realization of PBI is not available anymore in PCBSD.

I've searched a little bit for another solutions and it seems that two other systems exist actually - 0install and GNU Guix. So with the old PBI I've just explained there are three systems of software isolation that potentially can be used on FreeBSD. Some time ago 0install was even present in ports tree, I don't know why it was deleted.

Sorry for long post, but this problem is really important for me, I would like to see some answers of FreeBSD devs about the subject.
 
Since I am building my packages with ports-mgmt/poudriere, updating my FreeBSD Desktop is far easier. ports-mgmt/poudriere consistently rebuilds ports with all the needed dependencies.
My packages repository is created at the end of process only if there hasn't been any errors during the built of all my ports.
I can set my own build options for all the ports I need, and upgrade is as easy as :
# pkg upgrade
All my FreeBSD workstation are using the same validated package repository which is shared with NFS (can also be done with http server).
 
Peter2121 said:
dlegrand,
And what do you do if the users detect some problems with a package after the upgrade to the last version?
Click to expand...
You can keep old release of packages repository for rollback.
This can be set in /usr/local/etc/poudriere.conf
Code: 
# Keep older package repositories. This can be used to rollback a system
# or to bisect issues by changing the repository to one of the older
# versions and reinstalling everything with `pkg upgrade -f`
# ATOMIC_PACKAGE_REPOSITORY is required for this.
# Default: no
#KEEP_OLD_PACKAGES=no

# How many old package repositories to keep with KEEP_OLD_PACKAGES
# Default: 5
#KEEP_OLD_PACKAGES_COUNT=5
 
SirDice,
Using this 'stable' tree means that:
- one should not use packages
- one would stay on old versions of ALL ports
Once again, the problem is that on desktop system we often need to update one package to the last version and to downgrade another one to a previous one. Using ports on desktop system is hard, and the problem of coexistence of different versions of packages is NOT solved by using the 'stable' version of ports tree.

dlegrand,
As I understand, old packages are in old versions of repository. So again, we can downgrade ALL packages or upgrade ALL packages. This is not a feature I'm searching for.

abishai,
Sometimes, old packages can help. Sometimes they are in conflict with another packages (same dependencies of different versions). So it is not a good solution.
ports-mgmt/portdowngrade seems to be the best solution today, but there are two problems with it:
- difficult to use (one needs to search commits to revert)
- only ports can be used (no packages)
So in real life of desktop user ports-mgmt/portdowngrade is not an option...
 
Peter2121 said:
As about possible solutions - I think that coexistence of several ports in the ports tree is a possible solution, but definitely NOT the best one. Another solution should be used, based on isolation of software.
Click to expand...
Isolation of software is definitely an interesting concept Peter, but I have no idea how this could be implemented/integrated easily in the existing FreeBSD structure. Some kind of lite virtual machine or lite jail?

I believe that there's a Linux flavor which stores every dependent tools & libraries in a sub-directory of an application that requires them. That's interesting too, but not very convenient if you want to keep the source code as well.

PS: Thanks to the kind soul who edited my port tags. I used lots of them just to add a little color, but forgot that they also serve a purpose... :)

Dominique.
 
Peter2121 said:
Using this 'stable' tree means that:
- one should not use packages
- one would stay on old versions of ALL ports
Click to expand...

In the quarterly branch there are packages available, even for security updates:
http://pkg.freebsd.org/freebsd:10:x86:64/quarterly/All/

This is true you will stay with "old" versions, but you will face the same situation with most Linux or BSD desktop systems. And, as this branch gets updated every 3 months, the packages are still pretty new.

Fossette, don't forget that one should not update packages blindly:
http://svnweb.freebsd.org/ports/head/UPDATING?view=markup

Maybe, you could have solved your problem by installing another browser instead of trying to update Firefox? But again, reading the link above is mandatory, especially if you're tracking the last packages branch.

I stopped using FreeBSD as a desktop because of the then packages rolling release model. Too many of them were broken for days or weeks. There was no quarterly branch at this time which the aim is to avoid this kind of problem and which became the default repository lastly.

I am curious to have any feedback about it.
 
formateur_fou said:
This is true you will stay with "old" versions, but you will face the same situation with most Linux or BSD desktop systems.
Click to expand...
Yep, like on RHEL/CentOS 7. It's using Perl 5.16. Which was deprecated more than a year before RHEL7 was released.
 
formateur_fou said:
I stopped using FreeBSD as a desktop because of the then packages rolling release model.
Click to expand...
or
I stopped using FreeBSD
Click to expand...
is the fundamental problem to address. Migrants fleeing to other OS. I have another problem with my FreeBSD web browsers (a new forum post later), so I use firefox in my almost-1-year-old Windows XP VirtualBox guest. How sick is that?

My ports handling suggestion is not perfect and requires discipline, but I believe it's good for the road, it's simple to implement, and it eases the pain of both users and maintainers. It deserves some consideration at the next FreeBSD meeting.

Dominique.
 
Maintainers try to do it best here. I don't have any problem to use FreeBSD as a desktop. Currently, I'm using GNOME 3 and it works almost flawlessly. So I encourage people to try FreeBSD as a desktop, we have a lot of DEs to try.

Long Live to FreeBSD ;)
 
fossette said:
Migrants fleeing to other OS
Click to expand...
It was not my intention to discourage you.

I kept FreeBSD a long time on my desktop and I found it a very good system for my needs. I only had the feeling that sometimes, packages were not working as predictably as I would have wanted them to but that was not a big deal.

Over time, the new pkg tools made life much easier, and I guess the new repository will bring its own improvements too. This is why I am interested in getting feedback from people using it.
 
Peter2121 said:
SirDice,
Using this 'stable' tree means that:
- one should not use packages
- one would stay on old versions of ALL ports
Once again, the problem is that on desktop system we often need to update one package to the last version and to downgrade another one to a previous one. Using ports on desktop system is hard, and the problem of coexistence of different versions of packages is NOT solved by using the 'stable' version of ports tree.
Click to expand...

  1. There's a quarterly repository available for packages. You just need to edit /etc/pkg/FreeBSD.conf and edit the repository URL to read "quarterly" rather than "latest." [1]
  2. While a desktop user is likely to have more ports/packages installed than a server user, and so would certainly have more ports/packages to update, I have to wonder just how often you think these updates really need to take place. Is an absolute maximum of three months really too long to go without getting the latest version of a port? That still gets you the latest version of a package (though not necessarily a port) much sooner than OpenBSD gets it, not to mention just about any Linux distribution you can name. FreeBSD ports tend to be fresher than Gentoo packages; the Arch Linux devs have a much, much smaller number of packages to deal with, as do Exerbo and Alpine; CRUX will get you the latest version as soon as it comes out, in a manner of speaking: it has no official repository to speak of, as users make their own ports. In short, with the FreeBSD quarterly tree/repo, the inconvenience of waiting is negligible in my opinion.
  3. The quarterly ports tree and package repository still gets security updates and critical bug fixes. That hasn't been overlooked.
Personally, I switched to the quarterly ports tree because I build lots of ports with custom options using ports-mgmt/poudriere, but don't want to have to rebuild lots of them (or larger ports like editors/libreoffice) more often than necessary. At the same time, I don"t want to have to deal with keeping track of ignored/locked ports/packages, and don't want to eat up disk space with lots of backup packages I may never, ever need. There's no need to frequently build the latest ports available, and frankly, there's no way I or anyone else is missing out on something absolutely vital by having to wait a few weeks to a month before getting the latest release of something. I know that, because that's what the vast majority of *nix users do anyway. (I've also gathered in my time here that most FreeBSD users tend to build ports when first installing a system, then update at some lengthy interval--say when a bug fix is released, or manual intervention is needed, or maybe on a monthly basis.)

[1]: Sorry, I'm not at my machine right now, and I can't recall the key for that line.
 
ANOKNUSA, personally, I tend to adhere to the if-it-aint-broke-dont-fix-it philosophy. My WindowsNT guest VM has never been upgraded, and I'm quite ok with that. Lol!!! I like stability! An upgrade to one of the compilers? Not convinced it will have a significant impact on the ports already installed and running. My Firefox 38.0 had issues, but I kept it 4 months nonetheless. I don't have USB connection in VirtualBox, but I'll probably keep this 4.3.28 version until the next major release. I can do without the USB devices for now. I just don't want to risk it with upgrades, and spare me the wasted time should problems occur.

Dominique.
 
Guys,

Probably, there is a real problem of communication here. Look at the beginning of the thread. Two users (fossette and me) explained a problem of using FreeBSD as a desktop because of impossibility to install a previous version of software, different from the current one (present in ports/packages). We proposed to discuss a possible solution of this problem.

The ONLY post with a solution proposed was from abishai, who proposed to use ports-mgmt/portdowngrade. All others proposals are about using another repositories, another port trees or about update policy.

OK, guys, I'm really happy to get all these answers, but the question was definitely NOT about it! Using another repo/tree means synchro all ports/packages with the repo/tree, and the initial question was about the coexisting of different versions in the same repo/tree...
 
Indeed, you can downgrade ports but it's dangerous due the old versions contain bugs or lack some implementations as well.

Are you completely sure that you need/want this? In this case, you can still use an old release and you can downgrade the ports tree to its corresponding version, download it from FreeBSD archive site. Of course, all this at your own risk :)
 
cpm said:
I don't have any problem to use FreeBSD as a desktop. Currently, I'm using GNOME 3 and it works almost flawlessly. So I encourage people to try FreeBSD as a desktop, we have a lot of DEs to try.
Click to expand...
Ditto.

I don't get it. Sure, I occasionally have a hiccup when I upgrade a port but, otherwise, I've been running a FreeBSD desktop for 11 years! What's the big deal? It's not hard. Everything works. And I work on my workstation possibly longer in a day than anyone else.

I just don't get it.
 
Just to get back at the issue, I would recommend building your ports using the aforementioned ports-mgmt/poudriere. If you configure poudriere to use SVN to update the ports tree it's fairly easy to downgrade a single port by simply reverting that specific port to a previous commit. If you then build everything again poudriere will make sure the dependencies stay intact. But, use at your own risk of course. Some ports are updated due to security issues and by keeping that port on a vulnerable version you are opening the door for abuse.
 
Peter2121 said:
ports-mgmt/portdowngrade seems to be the best solution today, but there are two problems with it:
- difficult to use (one needs to search commits to revert)
- only ports can be used (no packages)
So in real life of desktop user ports-mgmt/portdowngrade is not an option...
Click to expand...
It is better than nothing and it works. I have used that in the past. Currently I follow a different solution. I have a second HDD in my PC for some stuff. From time to time I restore the latest backup to one partition and edit /etc/fstab. And of course I try to boot from that HDD. This should work and it gives the good feeling that recovery works - just in case. But additionally I have an old version of /usr/ports/ from the time of the last recovery. In the very seldom case that a port compiles well but does not work I simply copy the /usr/ports/whatever/it/is/ stuff from the backup HDD and re-compile that port.

This works with ports only but avoids to have to dig the correct SVN version. But basically this is just a side effect of the idea to verify the backups from time to time. Or it is an additional reason to do that.
 
I'd like to mention if there is a problem in an updated port, filing a problem report is the way to get these types of problems fixed faster. If you absolutely can't wait for the port to be fixed and ZFS is an option, another option you have is setting up your FreeBSD installation with ZFS to use boot environments. You can then just create a new boot environment, update your ports, and test. If there is a problem or regression, you can boot into the previous boot environment and delete the testing environment bringing you back where you started. Of course this assumes you are somewhat familiar with ZFS are or willing to become familiar with it. This is how I update when using portmaster(8) to update ports. When using ports-mgmt/poudriere as SirDice suggested, I run into much fewer issues with ports on other machines to begin with.
 
Ok, probably one more example would be nice here, just to help understanding the issue I often have with the current pkg system.
Today I need to add freerdp to my laptop. So pkg install freerdp should do it:
Code: 
[root@pcbsd-peter] ~# pkg install freerdp
Updating pcbsd-major repository catalogue...
pcbsd-major repository is up-to-date.
All repositories are up-to-date.
The following 50 package(s) will be affected (of 0 checked):
Installed packages to be REMOVED:
  libreoffice-4.3.7
  mupdf-1.7,1
New packages to be INSTALLED:
  freerdp: 1.2.0_5
  jpeg-turbo: 1.4.1
  xf86-video-scfb: 0.0.4_2
Installed packages to be UPGRADED:
  xorg-server: 1.14.7_5,1 -> 1.14.7_6,1
  libressl: 2.2.1 -> 2.3.0
  kdelibs: 4.14.3 -> 4.14.3_1
  curl: 7.43.0_2 -> 7.44.0
  openldap-client: 2.4.41 -> 2.4.42_2
  cyrus-sasl: 2.1.26_9 -> 2.1.26_12
  sylpheed: 3.4.3 -> 3.4.3_1
  ruby: 2.0.0.645,1 -> 2.0.0.647,1
  efl: 1.13.2 -> 1.15.1
  libarchive: 3.1.2_2,1 -> 3.1.2_4,1
  openvpn: 2.3.7 -> 2.3.8
  libtorrent-rasterbar: 1.0.4 -> 1.0.6
  py27-libtorrent-rasterbar: 1.0.4 -> 1.0.6
  gnome-vfs: 2.24.4_3 -> 2.24.4_4
  git: 2.4.6 -> 2.6.0
  p5-Net-SSLeay: 1.70 -> 1.72
  libssh2: 1.4.3_5,2 -> 1.6.0_1,2
  unrar: 5.21_1,5 -> 5.30,5
Installed packages to be DOWNGRADED:
  stunnel: 5.19 -> 5.14
Installed packages to be REINSTALLED:
  python27-2.7.10
  ptlib-2.10.10_3
  pcbsd-base-1425064224 (direct dependency changed: xf86-video-scfb)
  heimdal-1.5.3_4 (needed shared library changed)
  py27-cryptography-0.8.2 (needed shared library changed)
  serf-1.3.8 (needed shared library changed)
  apr-1.5.2.1.5.4 (needed shared library changed)
  nginx-1.8.0_3,2 (options changed)
  wget-1.16.3 (needed shared library changed)
  opusfile-0.6_1 (needed shared library changed)
  py27-curl-7.19.5.1 (needed shared library changed)
  pulseaudio-6.0_2
  trousers-tddl-0.3.10_7 (needed shared library changed)
  virtualbox-ose-4.3.30 (needed shared library changed)
  qca-2.1.0_3 (needed shared library changed)
  libvncserver-0.9.9_11 (direct dependency changed: jpeg-turbo)
  libevent2-2.0.22_1
  tor-0.2.6.9 (needed shared library changed)
  clamav-0.98.7 (needed shared library changed)
  postgresql93-client-9.3.9 (needed shared library changed)
  gkrellm2-2.3.5_6 (needed shared library changed)
  librtmp-2.4.20130923 (needed shared library changed)
  x11vnc-0.9.13_2 (direct dependency changed: libXrender)
  rdesktop-1.8.3 (needed shared library changed)
  socat-1.7.3.0_2 (needed shared library changed)
  p5-Crypt-OpenSSL-Random-0.10 (needed shared library changed)

The operation will free 399 MiB.
139 MiB to be downloaded.

Do I want to remove libreoffice-4.3.7?? NO!!!
Do I want to update xorg-server?? NO!!!
I just want to install freerdp because I need it NOW!
But the current pkg system does not allow me to install the freerdp package without updating some important parts of my system. This is a hell, I don't want to do all the updates now, I have no time, I've just asked to install freerdp!
 
Well, you're supposed to keep your system up to date for these reasons. Some updates are required for security and compatibility. It's like complaining about running FreeBSD 5.0 and not being able to install something until you update to version 10.2. That's not the fault of the ports tree (the subject of this matter, not packages but I haven't re-read through this whole thing).
 
drhowarddrfine,
My system is almost up-to-date, I've updated all packages three or four month ago approximately.
Please, explain me why must I update or remove LibreOffice (BTW, the last version does not work correctly on 10.1) when I just need to install freerdp? Why does my Mac OSX not ask me the things like this? Why does my Windows not ask me neither?
 
You must log in or register to reply here.
Back
Top