jails Issues with vnet Jail + IPv6

[Blue|Fusion]

Hello all,

I have been undergoing a process of moving my IPv4/6 configurations from the Jail /etc/rc.conf files to the /etc/jail.conf files for [theoretical] ease of maintenance. I am running into a problem, however.

If I leave it to the Jail's /etc/rc.conf to start the networking with the following the IPv6 DAD finishes basically instantly and services can bind to the assigned ULA, a SLAAC ULA and a SLAAC GLA no problem.

ifconfig_vnetns1="inet 172.23.1.10/24"
ifconfig_vnetns1_ipv6="inet6 fd33:58bc:59a0:2301::10/64 accept_rtadv"
defaultrouter="172.23.1.1"

However, when moving this functionality into /etc/jail.conf with the following exec.start sequence, IPv6 DAD takes several minutes and services such as ISC BIND do not bind the the IPv6 addresses after they are added:

path = "/jails/${name}";
host.hostname = "${name}.myfqdn.com";
$macprefix = "02:14:a4:21";
enforce_statfs 1;
mount.devfs;
exec.clean;
exec.consolelog = "/var/log/jails/${name}.log";
vnet;
vnet.interface = "vnet${name}";

# Default local FreeBSD pkg cache
mount = "/var/cache/pkg /jails/${name}/var/cache/pkg nullfs rw 0 0";

exec.prestart  = "ifconfig epair${id} create up";
exec.prestart += "ifconfig epair${id}a up descr jail:${name}";
exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
exec.prestart += "ifconfig epair${id}b ether ${macprefix}:${id}:0b";
exec.prestart += "ifconfig epair${id}b name vnet${name}";

exec.start  = "ifconfig vnet${name} ${ipv4} up";
exec.start += "route add default ${ipv4gw}";
exec.start += "ifconfig vnet${name} inet6 ${ipv6} up";
exec.start += "sh /etc/rc";

exec.poststart = "logger -t jail jail ${name} started";

exec.prestop = "ifconfig vnet${name} -vnet ${name}";

exec.stop = "sh /etc/rc.shutdown";

exec.poststop  = "ifconfig bridge${vlan} deletem epair${id}a";
exec.poststop += "ifconfig epair${id}a destroy";
exec.poststop += "logger -t jail jail ${name} stopped";

.include "/etc/jail.conf.d/*.conf";

/etc/jail.conf.d/ns1.conf

ns1 {
        $id = "52";
        $vlan = "101";
        devfs_ruleset = "7";
        $ipv4 = "172.23.1.10/24";
        $ipv4gw = "172.23.1.1";
        $ipv6 = "fd33:58bc:59a0:2301::10/64 accept_rtadv";
}

Why is IPv6 not working as expected when using jail.conf but is working quickly when using rc.conf?

 

[Blue|Fusion]

Upon watching this video I saw rtsol being used in the exec.start section and I tried it. It certainly helped, but still wasn't given enough time for the GLA IPv6 address to be setup before ISC BIND was loaded and bound to to the IP addresses. I inserted a sleep 1 after rtsol which did the trick!

Here is the updated /etc/jail.conf:

path             = "/jails/${name}";
host.hostname    = "${name}.myfqdn.com";
$macprefix       = "02:14:a4:21";
enforce_statfs 1;
mount.devfs;
exec.clean;
exec.consolelog  = "/var/log/jails/${name}.log";
vnet;
vnet.interface   = "vnet${name}";

# Default local FreeBSD pkg cache
mount            = "/var/cache/pkg /jails/${name}/var/cache/pkg nullfs rw 0 0";

# Destroy epair interfaces if they already exist
exec.prepare     = "(ifconfig epair${id}a && ifconfig epair${id}a destroy) || true";

# Create the epair, attach it to the proper VLAN bridge, set the MAC address, and rename the jail-side (b)
exec.prestart    = "ifconfig epair${id} create";
exec.prestart   += "ifconfig epair${id}a up descr jail:${name}";
exec.prestart   += "ifconfig bridge${vlan} addm epair${id}a";
exec.prestart   += "ifconfig epair${id}b ether ${macprefix}:${id}:0b";
exec.prestart   += "ifconfig epair${id}b name vnet${name}";

# Configure IPv4 and IPv6
exec.start       = "ifconfig vnet${name} ${ipv4} up";
exec.start      += "route add default ${ipv4gw}";
exec.start      += "ifconfig vnet${name} inet6 ${ipv6} up";
exec.start      += "rtsol vnet${name}";         # Required to get IPv6 configured right away
exec.start      += "sleep 1";                   # Give IPv6 router advertisements and DAD a moment to come up
exec.start      += "sh /etc/rc";

# Send a syslog message the jail is started (can this be smarter if failed?)
exec.poststart   = "logger -t jail jail ${name} started";

# Disconnect the epair from the jail
exec.prestop     = "ifconfig vnet${name} -vnet ${name}";

exec.stop        = "sh /etc/rc.shutdown";

# Clean up and send a syslog message
exec.poststop    = "ifconfig bridge${vlan} deletem epair${id}a";
exec.poststop   += "ifconfig epair${id}a destroy";
exec.poststop   += "logger -t jail jail ${name} stopped";

.include "/usr/local/etc/jail.conf.d/*.conf";

I would still appreciate feedback on making this process any cleaner, however.