jails Jail VNET VLAN

If I want to create jails using vlans, do I have to create a vlan on the host, so I will have 2 interfaces, create 2 bridges so as to assign the vnet to the necessary bridge, and use the host as a gateway to the internal network to route traffic between the jails?

My goal is to expose a virtual jail haproxy that redirects via reverse proxy the smb, syslog-ng ... services in virtual jail within a non-exposed internal network.
 
I don't understand that question, but that's because I don't see the purpose of using VLANs with jails ... so far I thought their only use was to separate traffic that shares the same physical medium (copper/fiber)? Or did I miss something here?
 
My goal is to expose a virtual jail haproxy that redirects via reverse proxy the smb, syslog-ng ... services in virtual jail within a non-exposed internal network.

The approach is not correct. I create an internal network and expose an instance that redirects the services.
 
If I understand that correctly, it has nothing to do with VLANs. But for a virtual internal network, sure you can build that on top of bridge(4) and epair(4), that's what I'm actually doing here at home.

(And I even do connect different virtual network segments to different physical network segments that still share the same switch, that's where VLANs come into play, but that doesn't seem to be part of your scenario.)
 
Understanding VLAN Configuration on FreeBSD by someone called genneko as well as reading vlan(4), epair(4), ifconfig(8) and jail(8) helped me a lot.

Edit: I realised that the below answers a question that was not specifically asked but it may still be useful.

Here is a rc.conf and jail.conf configuration that works for me. My re0 is physically connected to a switch's tagged port (aka trunk port) that is configured to accept VLAN 100 tags (and other VLAN tags).

Code:
### relevant rc.conf parameters
ifconfig_re0="up" # replace re0 with your interface
cloned_interfaces="re0.100"
ifconfig_re0_100="up"
# configure jail bridge/epair networking using exec.* parameters to avoid jail restart problems

jail_enable="YES"
jail_list="" # empty means all jails are started
jail_parallel_start="YES"

Code:
### basic VNET jail.conf configuration using VLAN 100 configured in the host's rc.conf (see above)
# I think vlan(4) tells you when exactly a VLAN tag is inserted; knowing this helped me troubleshooting
# Syntax error? Check if you forgot the trailing semi-colon.
# You may consider in jail.conf: .include “/etc/jail.conf.d/*.conf”;

vlan100jail { # note: ${name} will be auto-substituted with vlan100jail, see jail(8)
    path = "/jails/${name}"; # replace with the path to your jail
    host.hostname = "${name}";
    exec.clean; # clean environment

    mount.devfs; # check your jail's /dev directory to see what this parameter gives you there
    devfs_ruleset = 5; # see heading [devfsrules_jail_vnet=5] in /etc/defaults/devfs.rules

    vnet = new; # you can also set inherit here, see jail(8)
   
    vnet.interface = "epair0b"; # to be configured below using exec.*
    # note: I haven't tried but you can probalby just do vnet.interface = "re0.100"; and not use bridges/epairs
    # this should be equivalent to executing on the jailhost: ifconfig re0.100 vnet vlan100jail (see ifconfig(8))

    allow.socket_af; # optional; omitting this means being "restricted" to IPv4, IPv6, local (UNIX), and route
    allow.reserved_ports; # optional; omitting this means no access to ports lower than 1024

    # to make ping work:
    allow.raw_sockets;
    # also (took me hours to find that out): jexec vlan100jail sysctl net.inet.ip.fw.enable=0
    # otherwise the jail's firewall is active and you get "ping: sendto: Permission denied"

    # assuming neither bridge0 nor epair0 is present when jail starts
    exec.prepare += "ifconfig epair0 create up"; # note: you could also use = instead of += for the first occurrence
    exec.prepare += "ifconfig bridge0 create up";
   
    exec.prestart += "ifconfig epair0a up descr 'jailhost side of the epair'";
    exec.prestart += "ifconfig bridge0 up addm re0.100 addm epair0a descr 'bridge for the jail'";
   
    exec.start += "/bin/sh /etc/rc";
    # the vnet.interface parameter moves your interface from the host into the jail, so you can configure it here
    exec.start += "ifconfig epair0b 192.0.13.37/24 up descr 'jail side of the epair only visible in the jail'";

    exec.stop = "/bin/sh /etc/rc.shutdown";
   
    ### deletem-ing epair and re0.100 from bridge; otherwise cleanup (and restarting the jail) will fail
    exec.poststop += "ifconfig bridge0 deletem epair0a deletem re0.100";
    # remember: destroying either side of the epair deletes both, epair0a and epair0b
    exec.poststop += "ifconfig epair0a destroy";
    exec.poststop += "ifconfig bridge0 destroy";
}
 
So I created a vlan.10 I assigned an IP to the host's VLAN card - I created a bridge on it without assigning an IP and on it I created the jail pairs. Perfect by activating PF and NAT from vlan.10 to BGE0 the LAN also works navigation. Now if I try to activate a bridge on the LAN my entire rc.conf freezes

C-like:
ifconfig_bge0="inet 192.168.16.46 netmask 255.255.255.0"
defaultrouter="192.168.16.1"
cloned_interfaces="bge0.10 bridge10 bridge0"
ifconfig_bge0_10="inet 192.168.10.1 netmask 255.255.255.0"
ifconfig_bridge10="addm bge0.10 up"
gateway_enable="YES"

If I enter the following line everything freezes

C-like:
ifconfig_bridge0="addm bge0 up"
 
Your problem looks interesting but more information about your environment would help.

How is "everything freezes" reflected in your logs? Anything useful in /var/log/messages?

What is your reasoning behind assigning an IP address directly to bge0? Are you physically connected on the other side to a device that is not VLAN-capable? Is that device your uplink connection to the internet?

Configuring bge0 and bge0.10 the way you did and then putting both into the same bridge doesn't feel right. I'm guessing here a little how things are implemented to be honest but an interface is either tagged (accepting VLANs) or untagged (no VLAN). It is probably not supposed to be both at the same time. By the time you create bge0.10 you are in "tagged mode" and what happens if you still choose to configure more than up on bge0 in this case probably depends on the driver, bge(4). Perhaps this is the reason for "everything freezes". But again, I'm just guessing.

Did you try if the following from vlan(4) is applicable to your device? Perhaps
sysctl net.link.vlan.soft_pad=1 makes a difference.
vlan initially assumes the same minimum length for tagged and untagged
frames. This mode is selected by setting the sysctl(8) variable
net.link.vlan.soft_pad to 0 (default). However, there are network
devices that fail to adjust frame length when it falls below the allowed
minimum due to untagging. Such devices should be able to interoperate
with vlan after changing the value of net.link.vlan.soft_pad to 1. In
the latter mode, vlan will pad short frames before tagging them so that
their length is not less than the minimum value after untagging by the
non-compliant devices.
 
I only have one network interface - connected to a network that does not have VLANs or equipment that supports VLANs. So I just have bge0

I wanted to create 2 networks. one, vlan 10 would be internal, the jails run on it, for example samba syslog-ng dns server etc... the other network communicates with the other machines and on that one I wanted to use a host interface IP address for managing the jails etc. ... another haproxy IP address that acts as a reverse on the internal network. Currently the haproxy jail shares the network card with the host. I would have liked to use a vnet interface on the same network as the customer instead.

However bridge10 I'm creating on bge0.10 while bridge0 I'm creating on bge0. If I enter ifconfig_bridge0="addm bge0 up" I no longer contact the machine on the network I have to go into the console and remove bridge0
 
If your switch doesn't support 802.1q (vlans) and your router is not configured with vlan interfaces you can't use VLANs' to separate the network.

Instead use loopback interfaces for your internal network and bge0 bridge for external.

Edit:
when you have 802.1q on interface it's common practice to name vlan that is on this interface using interface_name.vlanid like bge0.10
when the vlans are internal only or on the hardware based switch cards those interfaces are named directly as VLAN10, VLAN20 etc.
if_bridge is software dump switch which doesn't understand VLANs so you can't add a trunk interface to it when the same interface have multiple vlans. If you have for example native vlan1 (untagged) +vlan10 and want to use them inside the bridge then you need to create 2 bridges one with bge0.1 and bge0.10 but you don't add the bge0 to any bridge as it will drop the traffic. So on your interface instead of configured the untagged traffic on vlan1 you create a vlan1 as bge0.1 and use it instead of bge0
 
Sorry me or bge0 I created bge0.10 vlan10 on which I created bridge10 and it works perfectly the vnets have no problem now I want to create bridge0 on bge0 because I want to create a machine on the same network as bge0.

So I use ifconfig_bridge0="addm bge0 up"

If I enter this option I completely lose Ethernet communication on bge0
 
I also tried this a few months ago, when I added a bridge on vlan 10 and "vlan 1" (default / untagged vlan), the host would freeze after a few hours.
Moving the IP address of the FreeBSD host from re0 to the "vlan 1" bridge didn't help either.
 
Disable all offloading (especially VLAN-related) on that realtek NIC, it's usually broken.
If this should run reliably also under load, don't use realtek - install a NIC from a proper vendor, e.g. Intel, Mellanox, Broadcom...

Also: bridging 2 VLANs is usually *never* a good Idea, especially the vlan1 which has many implications for switches. Basic rule for VLAN1: don't use it.
 
I mean two separate bridges, one on the interface itself and one on vlan 10.
Disabling VLAN offloading sounds like something I haven't tried before.
 
Back
Top