FreeBSD 15 Bridges, VLANs and Jails - Nice!

pprocacci said:
From what I gather from the information you provided em0 is a trunk.... so

1) Add em0 to bridge0 (w/out it having an address)
2) Create a vlan interface off of bridge0 (bridge0.10 for example)
3) DHCP on bridge0.10 (or whatever)

When you are done, it should look something like this (except I'm using ixl0 here):
ixl0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,HWSTATS>
ether 3c:fd:fe:aa:df:a8
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=10<VLAN_HWTAGGING>
member: ixl0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
port 1 priority 128 path cost 55 vlan protocol 802.1q untagged 1 tagged 10,20,30,40,50,60,70
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>

bridge0.10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1496
options=0
ether 58:9c:fc:10:6f:fa
inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
groups: vlan
vlan: 10 vlanproto: 802.1q vlanpcp: 0 parent interface: bridge0
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


If you want an rc.conf configuration here's mine. Substitute ixl0 and vlans as necessary:
ifconfig_ixl0="-txcsum -txcsum6 up"
ifconfig_bridge0="vlanfilter addm ixl0 tagged 10,20,30,40,50,60,70 untagged 1 up"
vlans_bridge0="10"
ifconfig_bridge0_10="SYNCDHCP up"

The above will send traffic on vlan 10 as you desire.
Click to expand...

pprocacci Thank you!

Unfortunately, bridge0.10 is not acquiring an ip address.

Here's my system. Note that I'm only using VLANS in the virtualbox guest os to manage jails. There is no upstream switch with VLAN capabilities in my setup.

0. LAPTOP : Freebsd 15.0-RELEASE : (my laptop DHCP's an address from my home router)
LAPTOP network
Bash: 
re0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether d4:93:90:3c:84:69
        inet 192.168.0.230 netmask 0xffffff00 broadcast 192.168.0.255
        inet6 fe80::d693:90ff:fe3c:8469%re0 prefixlen 64 scopeid 0x1
        inet6 2601:602:8100:d9a0:d693:90ff:fe3c:8469 prefixlen 64 autoconf pltime 304912 vltime 304912
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
arp -na
Bash: 
? (192.168.0.1) at a4:01:de:2f:38:95 on re0 expires in 1200 seconds [ethernet]
? (192.168.0.2) at 08:00:27:7f:a5:81 on re0 expires in 619 seconds [ethernet]
? (192.168.0.230) at d4:93:90:3c:84:69 on re0 permanent [ethernet]
netstat -arn
Bash: 
Internet:
Destination        Gateway            Flags         Netif Expire
default            192.168.0.1        UGS             re0
127.0.0.1          link#2             UH              lo0
192.168.0.0/24     link#1             U               re0
192.168.0.230      link#2             UHS             lo0

So the laptop looks good. The default route to my gateway is correct. Networking works fine. I can ping and ssh into the guest VM JAILHOST (server_a) at 192.168.0.2.

1. VirtualBox : Virtual Box is running on my laptop
2. JAILHOST (server_a): Freebsd 15.0-RELEASE : running inside virtualbox
JAILHOST network (with ifconfig_em0="DHCP")
Bash: 
hostname="server_a"
sshd_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
moused_nondefault_enable="NO"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
pf_enable="YES"
pf_fallback_rules_enable="YES"
pf_fallback_rules="block drop log all
pass quick on em4"        # multi-rule
pflog_enable="YES"
node_exporter_enable="YES"

syslogd_flags="-ss"
defaultrouter="192.168.0.1"

# jails
jail_enable="YES"
# TODO: is parallel start a good idea?
jail_parallel_start="YES"
gateway_enable="YES"
ipv6_gateway_enable="NO"

# <<<<<<<<<<<<<< INTERFACE SET TO USE DHCP >>>>>>>>>>>>>>>>>>>>>
ifconfig_em0="DHCP"

# <<<<<<<<<<<<<< VLAN STUFF DISABLED >>>>>>>>>>>>>>>>>>>>>
# Create bridge
#cloned_interfaces="bridge0"
#ifconfig_em0="-txcsum -txcsum6 up"
#ifconfig_bridge0="vlanfilter addm em0 tagged 10,20,30,40,50,60,70 untagged 1 up"
#vlans_bridge0="10"
#ifconfig_bridge0_10="DHCP"
#dhcp_timeout="60"
Bash: 
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=48505b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
    ether 08:00:27:7f:a5:81
    inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
arp -na
Bash: 
? (192.168.0.1) at a4:01:de:2f:38:95 on em0 expires in 1181 seconds [ethernet]
? (192.168.0.2) at 08:00:27:7f:a5:81 on em0 permanent [ethernet]
? (192.168.0.230) at d4:93:90:3c:84:69 on em0 expires in 506 seconds [ethernet]
netstat -arn
Bash: 
Internet:
Destination        Gateway            Flags         Netif Expire
default            192.168.0.1        UGS             em0
127.0.0.1          link#2             UH              lo0
192.168.0.0/24     link#1             U               em0
192.168.0.2        link#2             UHS             lo0

Looks good so far. The guest VM has arp entries for the default gateway and has full internet connectivity. Default gateway looks good.

But when I change JAILHOST to use VLANS, the ip address is not dhcp'd into bridge0.10 and there is no network and no connectivity.

3. JAILHOST with suggested changes
/etc/rc.conf
Bash: 
# ... <snip>
# 
# <<<<<<<<<<<<<< INTERFACE COMMENTED OUT >>>>>>>>>>>>>>>>>>>>>
# ifconfig_em0="DHCP"

# <<<<<<<<<<<<<< VLAN STUFF ENABLED >>>>>>>>>>>>>>>>>>>>>
# Create bridge
cloned_interfaces="bridge0"
ifconfig_em0="-txcsum -txcsum6 up"
ifconfig_bridge0="vlanfilter addm em0 tagged 10,20,30,40,50,60,70 untagged 1 up"
vlans_bridge0="10"
ifconfig_bridge0_10="DHCP"
dhcp_timeout="60"

Bash: 
sudo service netif restart

Networking is now broken:
* arp has no entries
* netstat just shows the localhost
*ifconfig shows that the bridge0.10 is missing an ipaddress (below)

Bash: 
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=48501b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
    ether 08:00:27:7f:a5:81
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33152
    options=0
    groups: pflog
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether 08:00:27:7f:a5:81
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=1<VLANFILTER>
    member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            port 1 priority 128 path cost 20000 vlan protocol 802.1q untagged 1 tagged 10,20,30,40,50,60,70
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0.10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1496
    options=0
    ether 08:00:27:7f:a5:81
    groups: vlan
    vlan: 10 vlanproto: 802.1q vlanpcp: 0 parent interface: bridge0
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

I must be missing something simple here. Any thoughts?
 
There is no upstream switch with VLAN capabilities in my setup.
Click to expand...

Right, that changes things. ;)

So talking strictly about the port connected to your switch, since it's not vlan aware, you just stick that port in bridge0 without any crazy options, and DHCP off of bridge0.
That'll give bridge0 an address that was gotten from your switch. You should be able to hit that ip without issue. That's the first part. If that doesn't work, don't continue until it does. ;)

Assuming that works, then you `tussle' with the remaining parts.

The first part to keep in mind is that your switch/router probably won't answer to broadcasts for which a vlan is set. The switch/router is "dumb".

So what you'll need to do is create a bridge0.10 (for example) and run dhcp on that interface for anything within that broadcast domain.
That'll allow anything sitting within vlan 10 to acquire addresses within that specific domain.
Add interfaces from your VMs on bridge0.10.

Once VMs are able to ping each other, it's time to get them out to the internet.
You need a way to forward packets to the "dumb" switch/router.
That can be done with PF and nat, forwarding packets that are not destined for the subnet you have associated with the VLAN.

That should be it.

If you're expecting VMs that are in a specific VLAN to be able to reach the switch and the switch respond to packets when a tag is set, my guess is it's just going to drop it entirely -- though this would be manufactuer specific.
This also means you need to setup the FreeBSD host (where the DHCP is) to be setup as a gateway. (gateway_enable)

---

There's an alternative to the above and in your case it might be the better choice.

Step one remains the same. Create bridge0, stick em0, and DHCP off of bridge0.

Step two however is creating a separate bridge for each domain you want to host behind it.
So bridge10, bridge20, etc. (note, no periods here)
You'd assign the first address of each broadcast domain on these interfaces, 192.168.10.1 on bridge10 and 192.168.20.1 on bridge20, etc....
Setup dhcp to respond to requests on bridge10, bridge20, etc....
Then setup the FreeBSD host to forward accordingly to the upstream as described above.

~Paul
 
You must log in or register to reply here.
Back
Top