Solved FreeBSD 14.2 Jail NullFS Mounts | Three Mounts For One Jail : mount -t nullfs [No such file or directory]

[rai.mohammed]

Hello every one;
I have a problem of mounting 3 hierarchical datasets (data17, base, pg_wal) with nested nullfs mount-points as explained in this Post thread
And to avoid a resource deadlock for mount_nullfs in my situation, I tried to stay away from hierarchical datasets with a layout like this :

zroot/jails/containers/pgdb01 -----------------------------#. mounted at /usr/local/jails/containers/pgdb01
zroot/jails/containers/pgdb01/data17 ----------------------#. mounted at /usr/local/jails/containers/pgdb01/data17 #(PG_DATA)
zroot/jails/containers/pgdb01/data17_base -----------------#. mounted at /usr/local/jails/containers/pgdb01/data17_base
zroot/jails/containers/pgdb01/data17_pg_wal ---------------#. mounted at /usr/local/jails/containers/pgdb01/data17_pg_wal


After creating the necessary directories (/var/db/postgres/data17/data17_base, /var/db/postgres/data17/data17_pg_wal) in the jail,
and adding the mount points in PostgreSQL jail config file /etc/jail.conf.d/pgdb01.conf :

# mount
mount += "${path}/data17 /var/db/postgres/data17 nullfs rw,local,noatime,nfsv4acls 0 0";
mount += "${path}/data17_base /var/db/postgres/data17/data17_base nullfs rw,local,noatime,nfsv4acls 0 0";
mount += "${path}/data17_pg_wal /var/db/postgres/data17/data17_pg_wal nullfs rw,local,noatime,nfsv4acls 0 0";


When I try to start the jail, I get an error preventing starting the jail - cannot start jail pgdb01 - mount_nullfs - mount -t nullfs [No such file or directory]

Now disabling the mount points in PostgreSQL jail config file /etc/jail.conf.d/pgdb01.conf , and starting the jail pgdb01 with no error .
You will notice the existing of the directories inside the jail pgdb01

• /var/db/postgres/data17/data17_base
• /var/db/postgres/data17/data17_pg_wal

 

[rafael_grether]

Lets do it from the beginning.

First, I will create the postgres datasets and mount them on the /var/db/postgres:


root@copernicus:~ # zfs create -o mountpoint=/var/db/postgres/ zroot/postgres
root@copernicus:~ # zfs create -o mountpoint=/var/db/postgres/data17 zroot/postgres/data17
root@copernicus:~ # zfs create -o mountpoint=/var/db/postgres/data17/data17_base zroot/postgres/data17/data17_base
root@copernicus:~ # zfs create -o mountpoint=/var/db/postgres/data17/data17_pg_wall zroot/postgres/data17/data17_pg_wall


So lets confirm that everything was created correctly:


root@copernicus:~ # zfs list -r zroot/postgres
NAME USED AVAIL REFER MOUNTPOINT
zroot/postgres 384K 118G 96K /var/db/postgres/
zroot/postgres/data17 288K 118G 96K /var/db/postgres/data17
zroot/postgres/data17/data17_base 96K 118G 96K /var/db/postgres/data17/data17_base
zroot/postgres/data17/data17_pg_wall 96K 118G 96K /var/db/postgres/data17/data17_pg_wall


Lets create some files to test (because I will not install postgres on my server now):


root@copernicus:~ # touch /var/db/postgres/data17/data17_base/databasefiles.dat
root@copernicus:~ # touch /var/db/postgres/data17/data17_pg_wall/moredatabasefiles.dat


Creating postgres jail pgdb01:

root@copernicus:~ # zfs clone zroot/jails/template_orig@default zroot/jails/pgdb01

Verifying the dataset was created:


root@copernicus:~ # zfs list zroot/jails/pgdb01
NAME USED AVAIL REFER MOUNTPOINT
zroot/jails/pgdb01 0B 118G 882M /usr/local/jails/pgdb01

Creating a simple jail.conf new entry with mount.fstab:


pgdb01 {
persist;
ip4 = inherit;
allow.raw_sockets;
mount.fstab = "/etc/fstab.pgdb01";
}


Next step, configure /etc/fstab.pgdb01 to mount_nullfs between datasets mountpoint and jail:


/var/db/postgres/data17/ /usr/local/jails/pgdb01/var/db/postgres/data17/ nullfs rw 0 0
/var/db/postgres/data17/data17_base /usr/local/jails/pgdb01/var/db/postgres/data17/data17_base/ nullfs rw 0 0
/var/db/postgres/data17/data17_pg_wall /usr/local/jails/pgdb01/var/db/postgres/data17/data17_pg_wall/ nullfs rw 0 0


Before start jail, I will create the directory on pgdb01 jail to receive nullfs mountpoints:


root@copernicus:~ # mkdir -p /usr/local/jails/pgdb01/var/db/postgres/data17/{data17_base,data17_pg_wall}


Starting the jail and checking nullfs mountpoint:


root@copernicus:~ # service jail start pgdb01
Starting jails: pgdb01.

root@copernicus:~ # jexec pgdb01

root@pgdb01:/ # ls -R /var/db/postgres/data17
data17_base data17_pg_wall

/var/db/postgres/data17/data17_base:
databasefiles.dat

/var/db/postgres/data17/data17_pg_wall:
moredatabasefiles.dat
root@pgdb01:/ #


So now, you can install postgresql on jail, create databases, and so on.
Thats it?

 

[rai.mohammed]

Hello M. rafael_grether , thanks for the response and the time spent in testing my use case.

It's my bad, after verifying my jail configuration pgdb01.conf, I noticed the error in the inversion between the target and the mount point.
I tried another test by mounting [/usr/ports, /var/cache/pkg] of the host inside the jail, and its work without error.

But in the situation where I will use the datasets for PostgreSQL DB, I preferred to adopt a native ZFS integration with Jail.
I wrote a small post explaining how to do a FreeBSD 14.2 Host-Managed ZFS datasets for PostgreSQL Jail,
where the datasets are auto-mounted With jexec inside a jail - native ZFS solution in this link.

Thank you again for you help.

 

[rafael_grether]

I personally don't like using ZFS integration with Jail, only in cases where it is strictly necessary, since I need to grant allow.mount, allow.mount.devfs and allow.mount.zfs permissions.

And by concept, I prefer to keep the jail as restricted as possible, which is why I prefer use nullfs in this case.

But I'm glad you made it.