Using pkg on Host to Manage Jails' Packages

[jef]

Objectives:

• Manage multiple jails' packages from the host environment
• Doing so should not require the jail to be active
• Doing so should not require the jail to have Internet connectivity
• Strongly desired that the jail's filesystem doesn't control the settings, or contain cached packages that are used
• Preferable if only a single copy of each package file is downloaded
• There is one, and only one way to manage the jails' packages to help reduce the chances of corruption or inconsistencies

Why this thread?

After a few years of trying to make consistent sense of the various options from the host as well as running pkg from a shell within the jail, I thought I had it understood, -r <root directory> was supposed to install at the specified root directory.

Unfortunately...

sudo pkg -r /var/jail/cloud install nginx failed to work as expected, both populating the host's root file system

$ ls -ld /usr/local/www
drwxr-xr-x  3 root  wheel  4 Dec 24 12:05 /usr/local/www
$ ls -lR /usr/local/www
total 1
lrwxr-xr-x  1 root  wheel  25 Dec 24 12:05 nginx -> /usr/local/www/nginx-dist
dr-xr-xr-x  2 root  wheel   2 Dec 24 12:05 nginx-dist

/usr/local/www/nginx-dist:
total 0

as well as failing to execute the install scripts with the errors

cp: /usr/local/etc/nginx/fastcgi_params-dist: No such file or directory
cp: /usr/local/etc/nginx/scgi_params-dist: No such file or directory
cp: /usr/local/etc/nginx/uwsgi_params-dist: No such file or directory
cp: /usr/local/etc/nginx/mime.types-dist: No such file or directory
cp: /usr/local/etc/nginx/nginx.conf-dist: No such file or directory

though these files were "properly" installed to the root as specified

$ ls -l /var/jail/cloud/usr/local/etc/nginx/
total 36
-rw-r--r--  1 root  wheel  1007 Dec 20 13:00 fastcgi_params-dist
-rw-r--r--  1 root  wheel  2837 Dec 20 13:00 koi-utf
-rw-r--r--  1 root  wheel  2223 Dec 20 13:00 koi-win
-rw-r--r--  1 root  wheel  5170 Dec 20 13:00 mime.types-dist
-rw-r--r--  1 root  wheel  2989 Dec 20 13:00 nginx.conf-dist
-rw-r--r--  1 root  wheel   636 Dec 20 13:00 scgi_params-dist
-rw-r--r--  1 root  wheel   664 Dec 20 13:00 uwsgi_params-dist
-rw-r--r--  1 root  wheel  3610 Dec 20 13:00 win-utf

So -r doesn't really restrict itself like pkg(8) indicates, "pkg will install all packages within the specified <root directory>."

Is this just a bug with the www/nginx package, or is it that packages, in general, are not robust to use of the -r option?

Edit: Looks like "POST-INSTALL" scripts fail rather often -- php72 as required by nextcloud72 fails repeatedly and it looks like the font cache was run on the host system's font cache, not that of the file system below the specified root.

Unfortunately, it seems as though the -c option will use the config files and cached packages in the target file system not the host's. Edit: It has additional problems, as it apparently uses the files in the chroot which apparently cause failures as the host system has different networking than the jail -- the jail's resolver is not reachable from the host (VIMAGE/vnet configuration).

$ sudo pkg -dc /var/jail/cloud/ install nginx
DBG(1)[11404]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[11404]> PkgRepo: verifying update for FreeBSD
DBG(1)[11404]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[11404]> Fetch: fetching from: http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly/meta.txz with opts "i"
DBG(1)[11404]> Fetch: fetching from: http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly/meta.txz with opts "i"
DBG(1)[11404]> Fetch: fetching from: http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly/meta.txz with opts "i"
pkg: http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly/meta.txz: No address record
[...]

Edit: Even with an in-chroot /etc/resolv.conf, the next problem is that the POST-INSTALL scripts use /dev/null, which isn't there on a non-running jail and can't be written on a locked-down root file system.

/bin/sh: cannot create /dev/null: Read-only file system
pkg: POST-INSTALL script failed

The -j option is a non-starter as it, as well as use of jexec, have both the problem of using the jail's config files and cached packages, as well as requiring Internet connectivity if using the FreeBSD binary-package repos. (I also seem to recall that -j requires the jail to be running, which is a challenge for "back-up" jails that shouldn't be running when the "primary" jail is running.)

I'd like to be able to manage the jails' contents from the comparatively "safe" environment of the host, without needing dump in new config and clean package caches (need depending on how tinfoil-hat one feels at the time).

Any suggestions of how to accomplish this?

 

[Yze]

Edit:

Hi jef,

have you tried using pkg -c <chroot path-to-jail>, that should execute pkg as host process (so you have connectivity) but execute POST-INSTALL scripts inside the chrooted enviroment (in that case your jail)

Okay now I understand with vnet environment that will cause additional issues. I did once encounter same issue and in my scripts I did install nginx via -c, e.g. snippet:

install_pkgs() {
  pkg -r ${JROOT} install -y pkg
  pkg -c ${JROOT} install -y nginx
}

but the environment does fit to the host so that is successful. You could possible nullfs the reqirements for e.g. nginx package installation, but for a "bullet proven" generic pkg installation you would need to actually start the jail...