jails [Solved] Jail Within Jail "jail_set: Operation not permitted"

doublerabbit

doublerabbit

So, I'm trying to setup a jail within a jail, or hierarchical jail, an undocumented feat by the looks of it.
After troubleshooting for hours on end, still not getting any success.

I have my host, I have my outer jail, and I've created my inner jail however when starting, I get the following message.

Code: 
Starting jails: cannot start jail  "web":
jail: web: jail_set: Operation not permitted

jail.conf
Code: 
infrastructure {
  # STARTUP/LOGGING
  exec.start = "/bin/sh /etc/rc";
  exec.stop  = "/bin/sh /etc/rc.shutdown";
  exec.consolelog = "/var/log/jail_console_${name}.log";

  # PERMISSIONS
  exec.clean;
  mount.devfs;
  enforce_statfs=0d;
  devfs_ruleset = 7;
  allow.vmm;
  allow.raw_sockets;
  allow.mount;
  allow.mount.devfs;
  allow.mount.procfs;
  allow.mount.fdescfs;
  allow.mount.linprocfs;
  allow.mount.zfs;
  allow.mount.nullfs;
  allow.mount.tmpfs;
  allow.raw_sockets;
  allow.socket_af;
  allow.sysvipc;
  allow.sysvipc = 1;
  allow.chflags;

  allow.socket_af = 1;

   # PATH/HOSTNAME
  path = "/forest/zone/infrastructure";
  host.hostname = "${name}";

  # NETWORKS/INTERFACES
  $id = "7";
  $ip = "x/27";
  $gateway = "x";
  $bridge = "bridge0";
  $epair = "epair${id}";

  # VNET/VIMAGE
  vnet;
  vnet.interface = "${epair}b";
 
  #
  exec.prestart += "ifconfig ${epair} create up";
  exec.prestart += "ifconfig ${epair}a up descr jail:${name}";
  exec.prestart += "ifconfig ${bridge} addm ${epair}a up";
  exec.created  += "zfs set jailed=on sas/zone/infrastructure";
  exec.created   = "zfs jail infrastructure sas/zone/infrastructure";
  exec.start    += "ifconfig ${epair}b ${ip} up";
  exec.start    += "route add default ${gateway}";

Code: 
security.jail.mount_linprocfs_allowed: 1
security.jail.mount_fdescfs_allowed: 1
security.jail.mount_nullfs_allowed: 1
security.jail.vmm_allowed: 1
security.jail.mount_zfs_allowed: 1
security.jail.mount_tmpfs_allowed: 1
security.jail.mount_procfs_allowed: 1
security.jail.mount_devfs_allowed: 1

I've copied the same config to my inner jail, but regardless of how complex or simple the inner jail configuration is, I still receive: Operation not permitted

Code: 
jail -v -c web
web: jail_set(JAIL_CREATE) persist name=web enforce_statfs=0 allow.vmm allow.raw_sockets allow.mount allow.mount.devfs allow.mount.procfs allow.mount.fdescfs allow.mount.linprocfs allow.mount.zfs allow.mount.nullfs allow.mount.tmpfs allow.socket_af=true allow.sysvipc=true allow.chflags securelevel=0 path=/crystal/zone/web host.hostname=web: Operation not permitted
jail: web: jail_set: Operation not permitted
or
Code: 
web: jail_set(JAIL_CREATE) persist name=web enforce_statfs=0 securelevel=0 path=/crystal/zone/web: Operation not permitted

I've trailed google & co, but cannot find anything on the matter.
Any help would be nice please.

Many Thanks

Edit: sigh
I was missing: children.max = xx
and mount.devfs was causing issues too.

Discovered via:
However is a jail within a jail suppose to be able to host devfs?
 
This is my /etc/jail.conf for a hierarchical jail a,
Code: 
# Common configs for all jails
path = "/jails/$name";
host.hostname = "$name";
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
persist;
ip4 = inherit;
ip6 = inherit;
mount.devfs;
mount.fdescfs;
allow.mlock;
allow.mount;
allow.mount.devfs;
allow.mount.fdescfs;
allow.mount.nullfs;
allow.mount.tmpfs;
allow.mount.procfs;
allow.mount.zfs;
enforce_statfs=1;
children.max=100;
allow.socket_af;
allow.raw_sockets;
allow.chflags;
allow.sysvipc;
a {
devfs_ruleset="20";
}
 
doublerabbit said:
However is a jail within a jail suppose to be able to host devfs?
Click to expand...
Not according to MWL's FreeBSD Mastery: Jails. Here's what he writes:
Code: 
mount.devfs;
devfs_ruleset="0";
The jails must have a devfs. Jails don't have access to the host's devd, though, and have no ability to create their own device filesystems.Their /dev is a copy of the parent jail's. Attempting to apply a ruleset is an error, though, so by using devfs_ruleset to specify ruleset 0 we tell jail(8) to not bother attempting to apply a ruleset.

It's worth getting hold of a copy of the book. It has an example of a hierarchical jail on page 174.
 
You must log in or register to reply here.
Back
Top