Jails, VNET, and VLAN help

B

Blue|Fusion

Hello all. I'm a new FreeBSD user and this is my first post here after a week of use.
I'm converting a Proxmox server over to FreeBSD and spinning up a bunch of jails for my core network services (DHCP, DNS, Unifi controller, Web, and HAProxy).

At present, my jails are working perfect with the exception of IPv6 connectivity and reading a bunch of tutorials and documentation leaves me confused on how to proceed.

I have multiple tagged VLANs trunked to the FreeBSD host and I am utterly confused how to setup VNET with them for IPv6 SLAAC support.

Code: 
root@host:/jails # jls 
   JID  IP Address      Hostname                      Path
    30  10.23.60.21     lb1.init-3.com                /jails/lb1
    31  10.23.60.51     mdb1.init-3.com               /jails/mdb1
    32  10.23.1.2       ns1.init-3.com                /jails/ns1
    34  10.23.0.40      unifi.init-3.com              /jails/unifi
    39  10.23.60.41     rdb1.init-3.com               /jails/rdb1

Host rc.conf:
Code: 
ifconfig_mlxen0="up"
ifconfig_mlxen1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport mlxen0 laggport mlxen1"

vlans_lagg0="2300 2301 2310 2315 2320 2330 2350 2360 2370"
ifconfig_lagg0_2300="up"
ifconfig_lagg0_2301="up"
ifconfig_lagg0_2310="up"
ifconfig_lagg0_2315="up"
ifconfig_lagg0_2320="up"
ifconfig_lagg0_2330="up"
ifconfig_lagg0_2350="up"
ifconfig_lagg0_2360="up"
ifconfig_lagg0_2370="inet 10.23.70.21 netmask 255.255.255.0"
ifconfig_lagg0_2370_ipv6="inet6 accept_rtadv"

defaultrouter="10.23.70.1"
jail_enable="YES"
jail_list="lb1 mdb1 ns1 rdb1 unifi"

Code: 
root@host:/jails # cat /etc/jail.conf
$jail_path = "/jails";
path = "$jail_path/$name";
host.hostname = "$name.init-3.com";

mount.devfs;
exec.clean;
exec.start = "sh /etc/rc";
exec.stop = "sh /etc/rc.shutdown";

jtemplate {
        ip4 = inherit;
        ip6 = inherit;
}

rdb1 {
        # Redis
        ip4.addr = "lagg0.2360|10.23.60.41";
}

unifi {
        # Unifi Controller
        ip4.addr = "lagg0.2300|10.23.0.40";
}

ns1 {
        # DHCP + DNS
        ip4.addr = "lagg0.2301|10.23.1.2";
        devfs_ruleset = "5";
}

lb1 {
        # HAProxy
        ip4.addr = "lagg0.2360|10.23.60.21";
}

mdb1 {
        # MariaDB Galera
        ip4.addr = "lagg0.2360|10.23.60.51";
}

web1 {
        # Nginx + PHP
       ip4.addr = "lagg0.2360|10.23.60.34";
}

Thank you in advance for any help and suggestions.
 
I have never used vlans with vnet jails, but I would guess that you have to create a vlan inside your jail and connect the other end of your epair to the bridge... .

# IPv6·
ifconfig_eth0_ipv6="inet6 accept_rtadv"
rtsold_enable="YES"

This is what I did with my vnet jails , but I had to deactivate it , because of a strange bug. I could never find the root cause , but the last time I enabled it was on 12.X not on 13.... . And if you have a static ipv6 prefix it should not be a problem. Not sure if I remember it correctly , but I my problem was rare and hard to reproduce. My assumption was sometimes when my isp rolled my dynamic ipv6 prefix( yes thanks deutsche Telekom ) it was not properly recognized by the jail.
 
Thank you, Sebastian for your response. So getting an IPv6 address inside of a Jail should be easy as you described with your rc.conf lines, but I believe I can't now until I get VNET set up. Maybe I don't even need VNET? I must use SLAAC due to the ISP assigned /56 does change on occasion (home use).

My goal is to keep VLANs separated at the host level as I presently have in rc.conf so a Jail has strict access to the VLAN(s) bridges I assign to it. Perhaps my understanding of VNET is incorrect, but I believe this is possible - I just can't find an example of the solution.
 
Well, I ended up finding the solution to the bridging and VNETs. Surprisingly not too difficult, but I would appreciate feedback if I can clean up the configuration any.

I had to set static MAC addresses for the Jail-side epairs for IPv6 SLAAC consistency across jail restarts. Again, if there's a better method, I welcome the feedback.

rc.conf on host:
Code: 
ifconfig_mlxen0="up"
ifconfig_mlxen1="up"
cloned_interfaces="lagg0 bridge2300 bridge2301 bridge2315 bridge2350 bridge2360"
ifconfig_lagg0="laggproto lacp laggport mlxen0 laggport mlxen1"
vlans_lagg0="2300 2301 2315 2350 2360 2370"
ifconfig_lagg0_2300="up"
ifconfig_lagg0_2301="up"
ifconfig_lagg0_2315="up"
ifconfig_lagg0_2350="up"
ifconfig_lagg0_2360="up"
ifconfig_lagg0_2370="inet 10.23.70.21 netmask 255.255.255.0"
ifconfig_lagg0_2370_ipv6="inet6 accept_rtadv"
defaultrouter="10.23.70.1"

#
# Bridges for jail VNETs
#
ifconfig_bridge2300="addm lagg0.2300 up"
ifconfig_bridge2301="addm lagg0.2301 up"
ifconfig_bridge2315="addm lagg0.2315 up"
ifconfig_bridge2350="addm lagg0.2350 up"
ifconfig_bridge2360="addm lagg0.2360 up"

/etc/jails.conf on host:
Code: 
# 1. definition of variables that we'll use through the config file
$jail_path = "/jails";
path = "$jail_path/$name";
host.hostname = "$name.init-3.com";

mount.devfs;
exec.clean;

#ip4 =inherit;
#ip6 =inherit;

exec.start = "sh /etc/rc";
exec.stop = "sh /etc/rc.shutdown";

# 7. specific jail configuration

jtemplate {
        $id = "0";
        $vlan = "2350";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}a -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

rdb1 {
        # Redis
        $id = "1";
        $vlan = "2360";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

unifi {
        # Unifi Controller
        $id = "2";
        $vlan = "2300";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

ns1 {
        # DHCP + DNS
        $id = "3";
        $vlan = "2301";
        devfs_ruleset = "5";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

lb1 {
        # HAProxy
        $id = "4";
        $vlan = "2360";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

mdb1 {
        # MariaDB
        $id = "5";
        $vlan = "2360";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}

web1 {
        #Nginx + PHP
        $id = "6";
        $vlan = "2360";

        vnet;
        vnet.interface = "epair${id}b";
        exec.prestart = "ifconfig epair${id} create";
        exec.prestart += "ifconfig epair${id}a up";
        exec.prestart += "ifconfig bridge${vlan} addm epair${id}a";
        exec.prestart += "ifconfig epair${id}b ether 02:14:e2:16:${id}:0b";
        exec.prestop = "ifconfig epair${id}b -vnet $name";
        exec.poststop = "ifconfig epair${id}a destroy";
}
 
You must log in or register to reply here.
Back
Top