Updating ports in 9.3-RELEASE

[zbang]

Hello,

I'm updating a system from 9.2 to 9.3. The base OS update was fine, but using portsnap to update the ports tree seems to take it into the realm of 10.0, not 9.3. One example:

--->  Checking for the latest package of 'devel/libffi'
--->  Found a package of 'devel/libffi': /usr/ports/packages/All/libffi-3.0.13.tbz (libffi-3.0.13)
--->  Fetching the package(s) for 'libffi-3.0.13_2' (devel/libffi)

where libffi-3.0.13_2.tbz does not exist in the i386/packages-9-current tree (libffi-3.0.13_1.tbz does).

I can set PKG_SITES to a particular place in the FTP tree, but the database/index will still point to newer versions. (This probably isn't helped by the lack of a packages-9.3-release directory on the FTP server and a related ports.tgz file.) What's the magic? I don't want to go to 10.0 since that'll entail some other work I'd like to avoid. OTOH I don't mind copying the packages directory, blowing away the entire ports tree, and recreating it.

Thanks,
z!
(who admits the problem may be the user...)

 

[talsamon]

PKG_SITES is deprecated. Delete it, if it exists in /usr/local/etc/pkg.conf. But libffi-3.0.13_2.txz exists.

 

[kpa]

The base OS update was fine, but using portsnap to update the ports tree seems to take it into the realm of 10.0, not 9.3.

There's only one ports tree and it's the same for all supported versions of FreeBSD. The differences if any are only realized when the ports are compiled and built into packages for a particular version of FreeBSD and a particular architecture.

 

[zbang]

But libffi-3.0.13_2.txz exists.

Thanks, but I'm not seeing that anywhere under ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/. That's a symptom; the bigger issue is why ports/packages are asking for it at all (this is not the only file where ports is looking for a newer file than is available from FTP).

Since I can, I'll try renaming the existing /usr/ports and recreate from the 9.3 ports.tgz file. Followed by running pkgdb -F.

It sounds like this doesn't affect ports, in that things resolve during build. I'm trying to update with packages.

 

[talsamon]

Make a file like this /usr/local/etc/pkg/repos/FreeBSD.conf

FreeBSD: {
#  url: "pkg+http://pkg.freebsd.org/freebsd:9:x86:64/latest",
#  url: "pkg+http://pkg0.bme.freebsd.org/freebsd:9:x86:64/latest",
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  enabled: yes
}

you can use the "ABI-line" or one of the two other commented out lines and you can use pkg install and pkg update.

https://wiki.freebsd.org/pkgng

 

[wblock@]

zbang, pkgdb is a program that comes with ports-mgmt/portupgrade. Mixing ports and packages successfully generally requires some experience, or it produces unexpected results.

I can't quite tell what you are trying to achieve. The Handbook has a chapter on ports and packages: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports.html. I have a short article on upgrading ports: Updating FreeBSD Ports.

 

[zbang]

zbang, pkgdb is a program that comes with ports-mgmt/portupgrade. Mixing ports and packages successfully generally requires some experience, or it produces unexpected results.

The goal is to update this system to 9.3 with all the 9.3 optional libraries/applications needed. IME it's occasionally necessary to mix ports and packages because one is down-rev compared to the rest of the ecosystem (there are very few source-built ports on this system, haven't needed to), and sometimes I have to build one or two things from their original source as the ports version is also down-rev from what I need. I don't consider this a problem, just something to work around. mplayer is a good example of this.

My usual method to update has been portupgrade -arRPP, filling in with -rRp if needed. I really don't want to compile all 450+ ports when packages are available for most of them (...x.org...).

Having renamed /usr/ports out of the way, untared a new copy from 9.3-release, copied the packages tree back into /usr/ports, I still see portupgrade looking for FTP directories that don't exist, example:

--->  Fetching ruby-1.9.3.484_2,1
fetch: ftp://ftp.FreeBSD.org//pub/FreeBSD/ports/i386/packages-9.3-release/All/ruby-1.9.3.484_2,1.tbz: File unavailable (e.g., file not found, no access)

This splits out into a couple of questions:
Should ports/i386/packages-9.3-release exist on the FreeBSD FTP server? If not, then what's the correct way to tell portupgrade to look elsewhere for the package files? Presumably at packages-9-current.

Is portupgrade deprecated? If so, is portmaster is the recommended replacement or is pkg?

If there is some specific knowledge needed to do this, I want to get that so I don't have to ask again. I've read both the handbook and wblock@'s page, and they generally posed more questions than they answered (for instance, about mixing ports/packages or using portupgrade).

 

[wblock@]

Ah, portupgrade's "pirate mode". -arR does not do anything different from just -a.

The contortions of moving port directories around are an indicator that either something is really wrong, or there is a basic misunderstanding. It looks like you are doing a lot of extra work to stick with the obsolete ports tree that shipped with 9.3. That's definitely a mistake if the object is to have updated ports.

ports-mgmt/portupgrade is not deprecated, but few recommend it any more. ports-mgmt/portmaster generally does the right thing without requiring Ruby, ruby-bdb, or a separate ports database that must be updated separately.

 

[kpa]

To use the official binary packages you don't need ports-mgmt/portupgrade, you can use just pkg-install(8), pkg-update(8) and pkg-upgrade(8) to install and update the packages.

The directory with the 9.3 packages you asked about is no longer supported, it was to be used with the old packaging tools (pkg_add etc.) that are now removed from use.

 

[youngunix]

This thread is a bit confusing!
Are you trying to upgrade 9.2 to 9.3-RELEASE? Or are you trying to update ports on 9.3-RELEASE?
If it is ports that needs updating, you can use devel/subversion, see sections 2-5 from this thread.

 

[zbang]

Posting an answer I started Friday--

The contortions of moving port directories around are an indicator that either something is really wrong, or there is a basic misunderstanding. It looks like you are doing a lot of extra work to stick with the obsolete ports tree that shipped with 9.3.

Moving the trees around was a simple way to make backups, that's all. My object is to have updated executables as if the system were newly installed as 9.3-RELEASE, be they compiled from source ("port") or pre-built ("package") with a preference for the latter. I have found from other painful experience that doing "just update to CURRENT" on a non-so-current system does not work well. Since 9.3 is shown as a Production Release, and the install tarballs are dated from >4 months ago, I'd hope that there is still some support for updating 9.3 systems.

ports-mgmt/portmaster generally does the right thing without requiring Ruby, ruby-bdb, or a separate ports database that must be updated separately.

Is that a recommendation to use portmaster over portupgrade?


Perhaps I'm being terminally dense, since when I read the docs, I don't see things that appear applicable. When the smoke clears, I'm looking for the correct mix of working commands that will update the optional software from that of a 9.2-RELEASE system to that of a 9.3-RELEASE system using as many packages as possible. Logic has already suggested that portupgrade with -RPP should do it, but evidently not. Portmaster has been suggested, but when I install portmaster via pkg_fetch/pkg_add, I get v3.17.3, and that instructs me to change over to PKGNG. Is that necessary? Desirable? Where are 9.3-RELEASE packages, anyway? In ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9-current/? (I really doubt that the 10.0 packages should be used on a 9.x system.)

So... should I:
Download/install pkg (which doesn't seem to be on this system, anyway)?
do the pkg2ng conversion or not?
Download/install portmaster or something else?
Point the process at packages-9-current (above), somewhere else, or let it default?
make distclean over the entire ports tree?
Something else?
(BTW- pkg-install, -update, and -upgrade aren't listed in INDEX-9, so I assume that's a typo)

I'm sorry that I sound frustrated, but well, I am. This seems like it should be a simply process, along the lines of " yum update", but it hasn't been so far. The actual O/S update was a pretty trivial process.

 

[wblock@]

Moving the trees around was a simple way to make backups, that's all. My object is to have updated executables as if the system were newly installed as 9.3-RELEASE, be they compiled from source ("port") or pre-built ("package") with a preference for the latter. I have found from other painful experience that doing "just update to CURRENT" on a non-so-current system does not work well.

"CURRENT" there does not mean what you think it means.

Since 9.3 is shown as a Production Release, and the install tarballs are dated from >4 months ago, I'd hope that there is still some support for updating 9.3 systems.

Well, yes. That has nothing to do with ports or packages. Unlike Linux, add-on applications really are separate. There is no 9.2 or 9.3 version of the ports tree. It is the same for all versions.

Is that a recommendation to use portmaster over portupgrade?

Yes.

Perhaps I'm being terminally dense, since when I read the docs, I don't see things that appear applicable. When the smoke clears, I'm looking for the correct mix of working commands that will update the optional software from that of a 9.2-RELEASE system to that of a 9.3-RELEASE system using as many packages as possible. Logic has already suggested that portupgrade with -RPP should do it, but evidently not. Portmaster has been suggested, but when I install portmaster via pkg_fetch/pkg_add, I get v3.17.3, and that instructs me to change over to PKGNG.

As of September, yes, "pkgng" (really just pkg()) is required.

Is that necessary? Desirable? Where are 9.3-RELEASE packages, anyway? In ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9-current/? (I really doubt that the 10.0 packages should be used on a 9.x system.)

I don't use packages because I don't have the patience for them. I do have the patience to wait for ports to compile, because then I don't have to mess with them. At present, I don't know if the "use packages if available" options of either portupgrade or portmaster work.

So... should I:
Download/install pkg (which doesn't seem to be on this system, anyway)?
do the pkg2ng conversion or not?
Download/install portmaster or something else?
Point the process at packages-9-current (above), somewhere else, or let it default?
make distclean over the entire ports tree?
Something else?

I'm sorry that I sound frustrated, but well, I am. This seems like it should be a simply process, along the lines of " yum update", but it hasn't been so far. The actual O/S update was a pretty trivial process.

You have a bunch of concepts mixed together. I'll explain this, but only in terms of ports because I don't use packages, and mixing the two requires some knowledge of both to do successfully. (In theory, pkg upgrade somewhat corresponds with yum update, but you must switch to pkg() first.)

0. Back up the system. Really.
1. Update the ports tree with portsnap() or svn(). (This gives the current version of the ports tree, which is the same for all supported versions of FreeBSD.)
2. Switch to pkg(). This is required because the old package system has ceased to exist. Specifically, install ports-mgmt/pkg and then run pkg2ng.
3. Install ports-mgmt/portmaster.

Those are the preliminaries. After that, I again refer you to my article, Upgrading FreeBSD Ports.

Sorry, I know you wanted to use packages where possible.

(BTW- pkg-install, -update, and -upgrade aren't listed in INDEX-9, so I assume that's a typo)

No, the man page name is pkg-install, but look at the contents. It talks about pkg install. The dash is just a concession to delimiters.

 

[jb_fvwm2]

Do you have /usr/sbin/pkg? With enough configuation, you could have a devel/subversion fetched ports tree and use a simple pkg upgrade or pkg install ... .... It works nicely here, most of the time, except the upgrades (if one uses pkg from the port and not base) whitch so far may not be error free. Portmaster is soon scheduled to acquire package capability again.

 

[SirDice]

Do you have /usr/sbin/pkg? With enough configuation, you could have a devel/subversion fetched ports tree and use a simple pkg upgrade or pkg install ... ....

There's no need to have a ports tree on /usr/ports/ if you use pkg(8). It works completely independent of it.

 

[jb_fvwm2]

FWIW, I three times ran pkg2ng on a machine within the last 24 hours, each time the resulting local.sqlite was huge, but showed no packages as installed. They were shown to be installed when deinstalling from the port itself. IIRC. Due to it being not the primary machine, I've already crafted a workaround that I emailed to the ports list, but it is going to take many hours longer, although only partly seamlessly. Hopefully someone fixes or debugs or seconds that problem. And decides to stick the workaround in one of the man pages as an EXAMPLE.

 

[ljboiler]

There's no need to have a ports tree on /usr/ports/ if you use pkg(8). It works completely independent of it.

Is that totally true? What about that very important UPDATING file that really needs to be consulted, as it contains important special instructions on upgrading things from time to time. Without the ports tree, that file is not available.

 

[kpa]

Is that totally true? What about that very important UPDATING file that really needs to be consulted, as it contains important special instructions on upgrading things from time to time. Without the ports tree, that file is not available.

That's the only exception at the moment because the UPDATING file is not available trough any other methods. You can however fetch your own copy of it independent of the ports update method used and direct pkg-updating(8) to use it:

svnlite export --force https://svn0.eu.freebsd.org/ports/head/UPDATING /tmp/UPDATING
pkg updating -f /tmp/UPDATING -d $(/bin/date -j -v -1m +%Y%m%d)

It would be very nice indeed if the PKG repository data included the UPDATING information as well. The other option that I would actually prefer would be to decouple the UPDATING information from the ports tree and migrate it to a similar database as the VuXML vulnerability database is.

 

[SirDice]

Is that totally true? What about that very important UPDATING file that really needs to be consulted, as it contains important special instructions on upgrading things from time to time. Without the ports tree, that file is not available.

You can always read it online: http://svnweb.freebsd.org/ports/head/UPDATING?view=log

The rest of the tree really doesn't have to be there. One company I work for has about 25-30 FreeBSD servers and the only server that has a complete ports tree is our build server. And it's not even needed there but it's convenient to have it.