Solved Jail VIMAGE

Ofloo

Ofloo

I've created bridge and epair interface:

Code: 
ifconfig bridge create
ifconfig epair create
ifconfig bridge0 addm re0 addm epair0a

Changed devfs_rules to:

Code: 
# Devices usually found in a jail.
#
[devfsrules_jail=4]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path zfs unhide
add path 'bpf*' unhide


Started the jail with current configuration:

Code: 
path = "/usr/jail/${name}";
host.hostname = "${name}.domain.net";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.start = "sleep 2";
allow.mount;
mount.devfs;

test {
  vnet;
  vnet.interface = "epair0b";
  devfs_ruleset = "4";
  exec.clean;
  exec.system_user = "root";
  exec.jail_user = "root";
  exec.start += "/bin/sh /etc/rc";
  exec.poststart = "/sbin/ifconfig lo0 up 127.0.0.1/8";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  persist;
}

But for some reason, .. the network can't go outside the jail why is this?

edit:
FreeBSD 10.1-RELEASE-p2 #3 r275861
 
My jail works for both vnet and non-vnet jails.

/etc/rc.conf
Code: 
# Network configuration
ifconfig_igb0="inet 192.168.1.100 netmask 255.255.255.0"
cloned_interfaces="lo1 bridge0"
ifconfig_bridge0="addm igb0"
ifconfig_lo1_aliases="inet 10.0.0.1-5 netmask 255.255.255.255"

# Jail
jail_enable="YES"
jail_list="samba serviio"

/etc/jail.conf
Code: 
# Jail configuration - /etc/jail.conf

allow.raw_sockets  = "1";
allow.set_hostname  = "0";
allow.sysvipc  = "1";
allow.mount.devfs;

host.hostname  = "${name}.local";
path  = "/tank/jails/${name}";
mount.fstab  = "/etc/fstab.${name}";
mount.devfs  = "1";
devfs_ruleset  = "4";
exec.consolelog  = "/var/log/jail_${name}_console.log";

# vnet jail with upnp-based dlna server with its own NIC address
serviio {
$if = "0";
$ip_addr = "192.168.1.102"; # Jail ip address
$ip_route = "192.168.1.100"; # Gateway or host's ip address
vnet;
vnet.interface = "epair${if}b";
exec.prestart = "ifconfig epair${if} create up";
exec.prestart += "ifconfig bridge0 addm epair${if}a";
exec.start = "/sbin/ifconfig lo0 127.0.0.1 up";
exec.start += "/sbin/ifconfig epair${if}b inet ${ip_addr} up";
exec.start += "/sbin/route add default ${ip_route}";
exec.start += "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.poststop = "ifconfig epair${if}a destroy";
persist;
}

# non-vnet jail for file sharing using internal address redirected by PF
samba {
  ip4.addr  = "10.0.0.1";
  interface = "lo1";
  exec.start  = "/bin/sh /etc/rc";
  mount  =  "procfs /tank/jails/samba/proc procfs rw 0 0";
}

Revision History:
21/12/14 - Added 'route add default' and cleaned up the script
 
Last edited:
Hmm, .... I've tried it, however for some reason it doesn't work, .. I'm curious could vlans have anything to do with it?
 
Ofloo said:
Hmm, .... I've tried it, however for some reason it doesn't work, .. I'm curious could vlans have anything to do with it?
Click to expand...

vnet jail works normal but non-vnet is having issues since its unreponsive to ping command.

Can you show me your modified jail.conf?
 
Removed my vlans, however still doesn't work.

rc.conf:
Code: 
cloned_interfaces="bridge0"
ifconfig_bridge0="addm re0"
jail_enable="yes"
jail_parallel_start="yes"
jail_list="jail1 jail2"

jail.conf

Code: 
path = "/usr/jail/${name}";
host.hostname = "${name}.domain.net";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.start = "sleep 2";
allow.mount;
mount.fstab = "/usr/jail/${name}/etc/fstab";
allow.set_hostname = 0;
allow.sysvipc = 1;
allow.raw_sockets = 1;
allow.socket_af;
mount.devfs = "1";
devfs_ruleset = "4";

jail1 {
  $if = "0";
  vnet;
  vnet.interface = "epair${if}b";
  exec.clean;
  exec.system_user = "root";
  exec.jail_user = "root";
  exec.prestart = "/sbin/ifconfig epair${if} create";
  exec.prestart += "/sbin/ifconfig bridge0 addm epair${if}a";
  exec.start += "/bin/sh /etc/rc";
  exec.poststart = "/sbin/ifconfig lo0 127.0.0.1/8 up";
  exec.poststart = "/sbin/ifconfig epair${if}b 192.168.1.15/24";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.poststop += "ifconfig epair${if}a destroy";
  persist;
}

jail2 {
  $if = "1";
  vnet;
  vnet.interface = "epair${if}b";
  exec.clean;
  exec.system_user = "root";
  exec.jail_user = "root";
  exec.prestart = "/sbin/ifconfig epair${if} create";
  exec.prestart += "/sbin/ifconfig bridge0 addm epair${if}a";
  exec.start += "/bin/sh /etc/rc";
  exec.poststart = "/sbin/ifconfig lo0 127.0.0.1/8 up";
  exec.poststart = "/sbin/ifconfig epair${if}b 192.168.1.15/24";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.poststop = "ifconfig bridge${if} deletem epair${if}a";
  exec.poststop += "ifconfig epair${if}a destroy";
  persist;
}

Code: 
jexec jail1 csh
# ifconfig epair0b 192.168.1.15/24
# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8<VLAN_MTU>
  ether 02:ff:50:00:06:0b
  inet 192.168.1.15 netmask 0xffffff00 broadcast 192.168.1.255
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
  status: active
# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
Killed

This or 100% loss.

Also the IPs in post start do not get assigned.
 
Makes no difference just tried it. If I may ask what version of FreeBSD are you using and what did you add/change to your kernel file?
 
I am using FreeBSD 10.1-STABLE and I only have VIMAGE added to the kernel.

I see your problem in your jail.conf

Ofloo said:
exec.poststart = "/sbin/ifconfig lo0 127.0.0.1/8 up";
exec.poststart = "/sbin/ifconfig epair${if}b 192.168.1.15/24";
Click to expand...

You forgot to add the '+' to the 2nd poststart line.

You should follow my example since it works and I don't use poststart.
 
Yeah I noticed however that was not the problem, I've finally found what the issue was. It's so silly, even though the network interface said it was active it required up in the config file.

Code: 
exec.prestart = "/sbin/ifconfig epair${if} create";

has to be

Code: 
exec.prestart = "/sbin/ifconfig epair${if} create up";

To me epair() has a bug, since it says it is UP while it's not!
 
That's good you found your minor mistake. I also solved my problem as well for non-vnet jails. I had to add one more line in Samba subroutine.

Code: 
interface = "lo1";

VIMAGE does look good but its still highly experimental. I hope it'll be stable enough to be used on production servers in next major release.
 
If I may, why would you need a non-vnet jail, .. you need the IP no matter what and you can add as much epairs as you might want?
 
Ofloo said:
If i may why would you need a non-vnet jail, .. you need the IP no matter what and you can add as much epairs as you might want?
Click to expand...

I won't have to worry about something like this when restarting non-vnet jail. It's good to have a clean jail restarts without FreeBSD complaining about something. vnet is good for firewall inside jail, upnp service and fewer restarts.

Code: 
Freed UMA keg was not empty (30 items). Lost 2 pages of memory.
Freed UMA keg was not empty (203 items). Lost 1 pages of memory.
hhook_vnet_uninit: hhook_head type=1, id=1 cleanup required
hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required
 
You must log in or register to reply here.
Back
Top