ZFS ZFS management inside jail

[patpro]

Hello,

I realize this kind of question has already been asked, but I was unable to find an answer to my problem in old posts or in google.

Background: I'm creating a web server where domain names must be absolutely isolated from each others. So I plan to create a jail for each domain/IP. Some of these domains are used by many people to publish web pages (think http://web/~login/).
I want to use ZFS inside each jail, so that I can :
- set quotas for each user/web site
- periodically create/destroy snapshots of each filesystem, so every user can browse past versions of h(is|er) own files

I've installed a brand new FreeBSD 10.1 Release server, installed ezjail, created my first jail.
I've set some sysctl on host:

security.jail.enforce_statfs=1 (tried with 0 too)
security.jail.mount_allowed=1
security.jail.mount_devfs_allowed=1
security.jail.mount_zfs_allowed=1

My first jail uses these settings /usr/local/etc/ezjail/perso_jail :

export jail_perso_jail_hostname="perso-clone"
export jail_perso_jail_ip="lo1|127.0.1.1,em0|PUBLIC-IP-ADDR"
export jail_perso_jail_rootdir="/Sites/perso-jail"
export jail_perso_jail_exec_start="/bin/sh /etc/rc"
export jail_perso_jail_exec_stop=""
export jail_perso_jail_mount_enable="YES"
export jail_perso_jail_devfs_enable="YES"
export jail_perso_jail_devfs_ruleset="devfsrules_jail"
export jail_perso_jail_procfs_enable="YES"
export jail_perso_jail_fdescfs_enable="YES"
export jail_perso_jail_image=""
export jail_perso_jail_imagetype="zfs"
export jail_perso_jail_attachparams=""
export jail_perso_jail_attachblocking=""
export jail_perso_jail_forceblocking=""
export jail_perso_jail_zfs_datasets="tank/ezjail/perso-jail/user"
export jail_perso_jail_cpuset=""
export jail_perso_jail_fib=""
export jail_perso_jail_parentzfs="tank/ezjail"
export jail_perso_jail_parameters=""
export jail_perso_jail_post_start_script=""
export jail_perso_jail_retention_policy=""

I've created a dedicated tank/ezjail/perso-jail/user for use inside my jail:

zfs create -o mountpoint=/user tank/ezjail/perso-jail/user
zfs set jailed=on tank/ezjail/perso-jail/user
zfs jail $JID tank/ezjail/perso-jail/user

Then I try to mount the new FS inside the jail:

ezjail-admin console perso-jail
zfs mount tank/ezjail/perso-jail/user
cannot mount 'tank/ezjail/perso-jail/user': Insufficient privileges

But it fails.

In the jail perso-jail, /user is supposed to hold about 1 hundred personnel filesystems, all mounted inside the jail as soon as this jail "starts". I need to be able to enforce quota (from outside the jail, if I've understood correctly the limitations). I need to be able to create/destroy snapshots inside the jail. And finally, each user needs to be able to access h(is|er) own snapshots by browsing ~/.zfs/snapshot/.
Of course, an intruder should not be able to access "foreign" zfs filesystems/datasets from within the jail.

What is the best way to achieve this?
thanks.

 

[patpro]

Ok. So as usual, I send a non-negligeable time writing my first post in a not-so-crappy english (being french does not help here), and eventually I fix it myself.
I've discovered that property jail_perso_jail_parameters in ezjail definition file /usr/local/etc/ezjail/perso_jail can be populated with this value: "enforce_statfs=1 allow.mount=1 allow.mount.zfs=1 allow.mount.procfs=1 allow.mount.devfs=1".
And automagically, everything goes fine:
- I see my ZFS dataset from inside the jail
- I can create/destroy children (filesystems or snapshots)
- an unprivileged user can automount snapshots by browsing some/.zfs/snapshot/foo path

Strangely, I've noted that it's not compulsory to use this command:

zfs jail $JID tank/ezjail/perso-jail/user

as, apparently, the following property in /usr/local/etc/ezjail/perso_jail has the exact same effect:

export jail_perso_jail_zfs_datasets="tank/ezjail/perso-jail/user"

But I'm not absolutely sure about that.

This is pretty encouraging, but I would be glad if someone with a trained eye would take a look at all this with security in mind. I'm a jail newbie and I'm a bit lost between vanilla jail and ezjail über-automation.

Thanks