Solved pkg system is seriously broken in a jail

klu

klu

I use ezjail to manage several jails. Now I've a problem where one of the jail's pkg system is completely broken, while other jails and the host OS's pkg works fine.

  • The host and all jails are on 11.3-RELEASE-p7 1103000
  • The pkg version on the broken jail is 1.13.2 (very strange)
  • The pkg version on the host OS and the rest of working jails is 1.14.5
  • `which pkg` on all jails show `/usr/sbin/pkg`
  • All jails `/usr/sbin` is mounted readonly from host's `/usr/jails/basejail` (how could different jails have different pkg version?)

Enough said, the surface of the problem is this: in the jail with broken `pkg`:

Code: 
root@media_server:~ # pkg install curl
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        curl: 7.69.1 -> 7.71.0

Number of packages to be upgraded: 1

Proceed with this action? [y/N]: y
[media_server] [1/1] Upgrading curl from 7.69.1 to 7.71.0...
[media_server] [1/1] Extracting curl-7.71.0:   0%
pkg: Fail to create /usr/local:Read-only file system  <<<<<<<<<<<<<<<<<
[media_server] [1/1] Extracting curl-7.71.0: 100%
 
getopt said:
In your jail it should look like this:
Code: 
> where pkg
/usr/local/sbin/pkg
/usr/sbin/pkg
/usr/local/sbin/pkg is installed after pkg-static bootstrap -f in a jail and creates userland ( /usr/local/). Every (Ez)jail has to have it's own userland.
Click to expand...
In the default PATH, base paths precede /usr/local, thus \which -a pkg
Code: 
/usr/sbin/pkg
/usr/local/sbin/pkg
and /usr/sbin/pkg calls the real one.
 
mjollnir said:
please supply the output of df
Click to expand...

So inside the broken jail, here's `df` output:

Code: 
root@media_server:~ # df /usr/sbin
Filesystem           1K-blocks     Used      Avail Capacity  Mounted on
/usr/jails/basejail 1861173660 13039212 1848134448     1%    [restricted]
root@media_server:~ # df /usr/sbin/pkg
Filesystem           1K-blocks     Used      Avail Capacity  Mounted on
/usr/jails/basejail 1861173620 13039212 1848134408     1%    [restricted]
 
mjollnir said:
Please post the output of mount from your base host system.
Click to expand...

That's it! The problem is that I mounted a unionfs to the jail's `/usr` directory read-only (but as "below"). After umount it, the problem is gone.

However, this is still not what I desire: The reason I unionfs /usr is because I've the jail's config file (/usr/local/etc/{x,y,z...}) somewhere in a git repo (`bsd_repo`). I made those config files available to the jail by unionfs this below the /usr mount. How come this made /usr/sbin readonly?

Code: 
>>> mount | grep media_server
mountdevfs on /usr/jails/media_server/dev (devfs, local, multilabel)
fdescfs on /usr/jails/media_server/dev/fd (fdescfs)
procfs on /usr/jails/media_server/proc (procfs, local)
/usr/jails/basejail on /usr/jails/media_server/basejail (nullfs, local, read-only)
<below>:/bsd_repo/src/jails/media_server/usr on /usr/jails/media_server/usr (unionfs, local, read-only)
~~~~~~^^^^
 
Thanks for all the help! I marked the thread as "solved", while I'll be researching a way to achieve what I want with union_fs
 
I have zroot/src/{13-CUR,12-STABLE,12.1-REL} ZFS datasets & this in my fstab(5):
Code: 
/src/13-CUR     /home/paul/Projects/FreeBSD/src/13-CUR unionfs  rw,late,below,noatime       0 0
/src/12-STABLE /home/paul/Projects/FreeBSD/src/12-STABLE unionfs rw,late,below,noatime 0 0
/src/12.1-REL /home/paul/Projects/FreeBSD/src/12.1-REL unionfs rw,late,below,noatime        0 0
It works fine, this way I keep my patches in my $HOME to overrule some shipped files. Maybe I'll try sysutils/fusefs-unionfs instead.
 
  • Like
Reactions: klu
You must log in or register to reply here.
Back
Top