Pkg orphaned even when it's availabe

[SirDice]

The new pkg-1.3.2 is causing me some problems. I have built a clean new set of packages with poudriere but pkg upgrade refuses to update itself. If I manually remove pkg and extract and use the new 1.3.2 pkg-static to add it, pkg correctly shows 1.3.2. But pkg version -vR shows it's "orphaned" even though it's available in the repository.

<...snip...>
phpMyAdmin-4.2.6                   =   up-to-date with remote
pkg-1.3.2                          ?   orphaned: ports-mgmt/pkg
png-1.5.18                         =   up-to-date with remote
<...snip...>

root@c2:~remko# pkg -vv
Version                 : 1.3.2
PKG_DBDIR = "/var/db/pkg";
PKG_CACHEDIR = "/var/cache/pkg";
PORTSDIR = "/usr/ports";
INDEXDIR = "";
INDEXFILE = "INDEX-9";
HANDLE_RC_SCRIPTS = false;
ASSUME_ALWAYS_YES = false;
REPOS_DIR [
    "/etc/pkg/",
    "/usr/local/etc/pkg/repos/",
]
PLIST_KEYWORDS_DIR = "";
SYSLOG = true;
ABI = "freebsd:9:x86:64";
DEVELOPER_MODE = false;
VULNXML_SITE = "http://www.vuxml.org/freebsd/vuln.xml.bz2";
FETCH_RETRY = 3;
PKG_PLUGINS_DIR = "/usr/local/lib/pkg/";
PKG_ENABLE_PLUGINS = true;
PLUGINS [
]
DEBUG_SCRIPTS = false;
PLUGINS_CONF_DIR = "/usr/local/etc/pkg/";
PERMISSIVE = false;
REPO_AUTOUPDATE = true;
NAMESERVER = "";
EVENT_PIPE = "";
FETCH_TIMEOUT = 30;
UNSET_TIMESTAMP = false;
SSH_RESTRICT_DIR = "";
PKG_ENV {
}
DISABLE_MTREE = false;
PKG_SSH_ARGS = "";
DEBUG_LEVEL = 0;
ALIAS {
}
CUDF_SOLVER = "";
SAT_SOLVER = "";
RUN_SCRIPTS = true;
CASE_SENSITIVE_MATCH = false;
LOCK_WAIT = 1;
LOCK_RETRIES = 5;
SQLITE_PROFILE = false;
WORKERS_COUNT = 0;


Repositories:
  tc-repo: {
    url             : "http://our-repo.example.com/packages/TC92-RELEASE-default/",
    enabled         : yes
  }

It's there:

root@c2:~# fetch http://our-repo.example.com/packages/TC92-RELEASE-default/All/pkg-1.3.2.txz
pkg-1.3.2.txz                                 100% of 1937 kB   94 MBps 00m00s

Edit, in case it's important:

root@c1:~# pkg version -vI | grep poudriere
poudriere-devel-3.0.99.20140626    =   up-to-date with index

 

[kpa]

There's some funkiness with ports-mgmt/poudriere-devel (I guess also with the non-devel version) about how the symbolic links in the repository directory are updated. Sometimes when you have a failed build the links are left in a non-consistent state and you'll see no updated packaged on pkg upgrade even if poudriere just told you that it built n packages. The only solution I've found so far is to delete the packages directory completely and build everything from scratch.

 

[SirDice]

The only solution I've found so far is to delete the packages directory completely and build everything from scratch.

I already did, twice

I also ran pkg repo -L /usr/local/poudriere/data/packages/TC92-RELEASE, that helped with some older pkg versions but then it upgrades everything except pkg itself.

 

[kpa]

Try running pkg update -f, I had to do that once to force ports-mgmt/pkg to really update its view of the repository metadata.

 

[SirDice]

Still thinks pkg is orphaned.

root@c2:~# pkg update -f
Updating repository catalogue
Fetching meta.txz: 100% of 292 B
Fetching digests.txz: 100% of 10 kB
Fetching packagesite.txz: 100% of 61 kB

Adding new entries: 100%
Incremental update completed, 206 packages processed:
0 packages updated, 0 removed and 206 added.
root@c2:~# pkg upgrade
Updating repository catalogue
tc-repo repository is up-to-date
All repositories are up-to-date
Checking for upgrades: 100%
Checking integrity... done (0 conflicting)
Your packages are up to date
root@c2:~# pkg version -vR | grep pkg
pkg-1.3.2                          ?   orphaned: ports-mgmt/pkg
root@c2:~#

I'm beginning to think pkg doesn't add itself to the repository files. All other files are picked up and installed correctly. I'm going to try a pkg upgrade -f.

 

[kpa]

Also make sure ports-mgmt/pkg is in your list of ports to be built by poudriere. It's not a dependency of any other port so it might be left out otherwise.

 

[SirDice]

I'm going to try a pkg upgrade -f.

This re-installed everything except pkg.

 

[SirDice]

Also make sure ports-mgmt/pkg is in your list of ports to be built by poudriere. It's not a dependency of any other port so it might be left out otherwise.

It's there, it's been built. It's available on the server. But pkg just refuses to see it.

 

[jrm@]

Have you reported the issue? https://github.com/freebsd/pkg/issues?state=open

 

[kpa]

Have you reported the issue? https://github.com/freebsd/pkg/issues?state=open

This is very likely a problem with Poudriere, it has its own routines to deal with the repository metadata instead of using just pkg-repo(8) so it doesn't have to leave the repository in a half finished state when a build fails.

One trick you can try is to run pkg repo manually on the repository directory: pkg repo /path/to/repo /path/to/signing_key

 

[kpa]

@SirDice, can you post the ls -al listing of your repository directory, the one that contains the All, Latest subdirectories and the digest.txz etc. files.

 

[SirDice]

One trick you can try is to run pkg repo manually on the repository directory:

pkg repo /path/to/repo /path/to/signing_key

Yes, I had read about the database changes. So I already ran pkg repo -L /usr/local/poudriere/data/TC92-RELEASE-default/. That helped with older versions of pkg but it still refuses to upgrade pkg itself. Even after I removed and installed it by hand.

root@c1:/usr/local/poudriere/data/packages/TC92-RELEASE-default# ll
total 4
lrwxr-xr-x  1 root  wheel   16 Jul 28 12:04 .latest -> .real_1406541840
drwxr-xr-x  4 root  wheel  512 Jul 28 12:04 .real_1406541840
lrwxr-xr-x  1 root  wheel   11 Jan  6  2014 All -> .latest/All
lrwxr-xr-x  1 root  wheel   14 Jan  6  2014 Latest -> .latest/Latest
lrwxr-xr-x  1 root  wheel   19 Jan  6  2014 digests.txz -> .latest/digests.txz
lrwxr-xr-x  1 root  wheel   16 Jul 28 11:19 meta.txz -> .latest/meta.txz
lrwxr-xr-x  1 root  wheel   23 Jan  6  2014 packagesite.txz -> .latest/packagesite.txz

And just for shit and giggles:

root@c1:/usr/local/poudriere/data/packages/TC92-RELEASE-default# ll All/pkg-1.3.2.txz
-rw-r--r--  1 root  wheel  1983712 Jul 28 11:32 All/pkg-1.3.2.txz

So it's there alright. I'm beginning to suspect it's somehow skipped when creating the repo files. Even if the file is there if it's not in the repo database the tool will simply state it doesn't exist.

 

[SirDice]

Have you reported the issue? https://github.com/freebsd/pkg/issues?state=open

Great.

https://bugs.freebsd.org/bugzilla/show_ ... ?id=192188

I'll create another one at github.

 

[SirDice]

I had a look at packagesite.yaml (extracted packagesite.txz), all files are there except an entry for pkg-1.3.2. So it looks like it's being skipped when creating the repo files.

 

[kpa]

Even with the 1.3.4 version there are still some problem with upgrades. I noticed that on my systems pkg-upgrade(8) keeps missing updated packages and also packages that have changed dependencies. PR at:

https://github.com/freebsd/pkg/issues/907

 

[SirDice]

My original issue seems to have disappeared. It's now able to see the package on the server and I'm able to upgrade pkg itself. Still not sure what happened though but today I updated pkg on the poudriere server itself first. Then cleaned out all the old package trees and built new ones. That seemed to have done the trick.

As for updating from pkg-1.2.x to pkg-1.3.x I did notice I had to pkg update -f after pkg was upgraded or the system wouldn't see the new packages that were available.

 

[kpa]

Version 1.3.5 of ports-mgmt/pkg was just released. It promises to fix the upgrade issues but I can't test anything yet because none of the ports I use have new updates at the moment.