jails thinjail nullfs mountpoint does not get unmounted on stop

P

pgzh1

Hi,
I have a small issue with thinjails set up according to the FreeBSD handbook ( https://docs.freebsd.org/en/books/handbook/jails/ ).

For some reason the root nullfs mountpoint of the jail does not get unmounted after stopping it with
Code: 
service jail onestop thinjail
Therefore the jail cannot be started again:
Code: 
# service jail onestart thinjail
Starting jails: cannot start jail  "thinjail":
mount_nullfs: /jail/nullfs-base/thinjail: Resource deadlock avoided
jail: thinjail: /sbin/mount -t nullfs -o ro /jail/templates/freebsd-base /jail/nullfs-base/thinjail: failed
.

My setup is the following:
Code: 
zroot/jail                                     884M   183G   158M  /jail
zroot/jail/containers                          352M   183G   112K  /jail/containers
zroot/jail/containers/thinjail                 188K   183G  5.41M  /jail/containers/thinjail
zroot/jail/templates                           374M   183G    96K  /jail/templates
zroot/jail/templates/freebsd-base              369M   183G   369M  /jail/templates/freebsd-base
zroot/jail/templates/freebsd-overlay          5.42M   183G  5.33M  /jail/templates/freebsd-overlay

Code: 
# ls -lh /jail/containers/thinjail/
total 35 KB
drwxr-xr-x  30 root wheel  108B Apr  6 14:57 etc
drwxr-xr-x   2 root wheel    2B Mar 26 20:36 home
drwxr-x---   2 root wheel    8B Apr  6 14:57 root
drwxr-xr-x  24 root wheel   24B Apr  6 14:57 tmp
drwxr-xr-x   3 root wheel    3B Mar 26 21:16 usr
drwxr-xr-x  24 root wheel   24B Nov 28 04:42 var

# ls -lh /jail/templates/freebsd-base/
total 78 KB
-r--r--r--   1 root wheel  5.9K Nov 28 04:51 COPYRIGHT
drwxr-xr-x   2 root wheel   49B Mar 26 20:30 bin
drwxr-xr-x  15 root wheel   71B Mar 26 20:30 boot
dr-xr-xr-x   2 root wheel    2B Nov 28 04:42 dev
lrwxr-xr-x   1 root wheel   11B Apr  4 17:08 etc -> overlay/etc
lrwxr-xr-x   1 root wheel   12B Apr  4 17:08 home -> overlay/home
drwxr-xr-x   4 root wheel   80B Mar 26 20:30 lib
drwxr-xr-x   3 root wheel    8B Nov 28 04:43 libexec
drwxr-xr-x   2 root wheel    2B Nov 28 04:42 media
drwxr-xr-x   2 root wheel    2B Nov 28 04:42 mnt
drwxr-xr-x   2 root wheel    2B Nov 28 04:42 net
drwxr-xr-x   2 root wheel    2B Mar 26 20:42 overlay
dr-xr-xr-x   2 root wheel    2B Nov 28 04:42 proc
drwxr-xr-x   2 root wheel  152B Mar 26 20:30 rescue
lrwxr-xr-x   1 root wheel   12B Apr  4 17:08 root -> overlay/root
drwxr-xr-x   2 root wheel  150B Mar 26 20:30 sbin
lrwxr-xr-x   1 root wheel   11B Nov 28 04:42 sys -> usr/src/sys
drwxr-xr-x   2 root wheel    2B Apr  6 13:33 tmp
drwxr-xr-x  13 root wheel   14B Apr  4 17:09 usr
lrwxr-xr-x   1 root wheel   11B Apr  4 17:08 var -> overlay/tmp

Code: 
# cat /etc/jail.conf.d/thinjail.conf
thinjail {
  # PERMISSIONS
  devfs_ruleset="5";
  enforce_statfs="1"; # needed to allow file system mounts below jail root
  allow.raw_sockets;
  allow.reserved_ports;
  mount.devfs;
 
  vnet;
  vnet.interface = "iavf0";
 
  # STARTUP/LOGGING
  exec.start = "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.poststop = "";
  exec.consolelog = "/var/log/jail_console_${name}.log";
  exec.clean;
 
  # HOSTNAME/PATH
  host.hostname = "${name}";
  path = "/jail/nullfs-base/${name}";
 
  # MOUNT
  mount.fstab = "/jail/${name}.fstab";
}

Code: 
# cat /jail/thinjail.fstab
/jail/templates/freebsd-base    /jail/nullfs-base/thinjail                      nullfs  ro                      0 0
/jail/containers/thinjail       /jail/nullfs-base/thinjail/overlay              nullfs  rw                      0 0
tmpfs                           /jail/nullfs-base/thinjail/tmp                  tmpfs   rw,mode=777,size=512M   0 0

Code: 
# cat /var/log/jail_console_thinjail.log
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Starting Network: lo0 iavf0.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
iavf0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether 3c:fd:fe:d5:98:f2
        inet 10.0.7.12 netmask 0xffffff00 broadcast 10.0.7.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.0.7.1
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating motd:.
Creating and/or trimming log files.
Clearing /tmp (X related).
Updating /var/run/os-release done.
Starting syslogd.
Starting cron.
 
Mon Apr  6 14:57:34 CEST 2026
Stopping cron.
Waiting for PIDS: 4933.
.
Terminated

This is a jail I just created from the template I made according to the handbook.

I know I could just use exec.prestop to unmount the nullfs base, but something in the process of stopping the jails seems to be wrong here.
Actually I have another jail in which I try to run nfsd which also does not stop as it should - that nfsd jail goes to the "dying" state when I try to stop it and does not release the vnet device, and I have to reboot the host to resolve this situation.
Since this basic jail already exhibits an issue when beeing stopped, I would like to resolve that first and see if it also resolves the issue with the other jail.

As always, any help would be very much appreciated!

Edit: I'm running FreeBSD 15.0-RELEASE-p5 .
 
You must log in or register to reply here.
Back
Top