ZFS Resource busy when unmounting jail fstab, ZFS and nullfs mount

Hi, I have troubles when stopping a jail (configured in jail.conf).
The jail root: I have an empty ZFS dataset, mounted readonly. On top of it I have a readonly nullfs mount containing the system files:

Empty ZFS dataset mounted in: .../myjail/mnt
Readonly nullfs mount on top: .../myjail/mnt
Additional nullfs mounts below the jail root.

When stopping the jail, the subdirectories get unmounted properly. Only the root (nullfs) cannot be unmounted:
Bash:
# jail -r -v myjail
...
myjail: run command as root: /sbin/umount .../myjail/mnt/dev/fd
myjail: run command as root: /sbin/umount .../myjail/mnt/dev
myjail: run command as root: /sbin/umount -t nullfs .../myjail/mnt/var
myjail: run command as root: /sbin/umount -t nullfs .../myjail/mnt/tmp
...
myjail: run command as root: /sbin/umount -t nullfs .../myjail/mnt/
umount: unmount of .../myjail/mnt failed: Device busy
jail: myjail: /sbin/umount -t nullfs .../myjail/mnt/: failed

lsof and fstat don't show any processes that block the directory, so it must be the kernel holding it.

My question: is the problem created by the identical mountpoint of the ZFS dataset and the nullfs mount?
What is the proper way to do it?

I use the read-only ZFS dataset to prevent writing to the "mnt" directory when the jail is not started/mounted. I could theoretically mount the dataset as read-only and use a sub-directory to mount the nullfs Jail root. Would this be the proper way?

Update: I tried to mount the nullfs root from a subdirectory of the ZFS dataset. Mountpoints are different but still no luck.
 
Update: I found this news thread: https://lists.freebsd.org/pipermail/freebsd-jail/2011-August/001603.html
It turns out, my jail is in state of dying and I can see this with jail -d. I have no idea why it does not want to die completely. Maybe something is happening with the VNET stack. Before stopping the jail I remove all network interfaces from its VNET and then destroy them. So it should not be a network issue in theory.
My current workaround is to unmount all jail's mounts in "exec.poststop". In case some mount is held, I forcibly unmount it (which is the case for the jail root). This works. The only issue is that the dying jails stay indefinitely :(
 
Back
Top