Bridge with VLAN and TAP members does not appear to pass traffic

S

StayFrosty

Hello,

I am setting up a VM host using bhyve and jails with the requirement that different VMs and jails live on a variety of VLANs. I have done similar setups using KVM and Xen in the past so my guess is my head is full of linuxisms and I'm approaching this the wrong way.

The server is connected to the switch via two gigabit NICs in an LACP configuration. This is a port-channel interface on the switch and is configured as a trunk with a native VLAN.
Code: 
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether d0:50:99:19:2a:ab
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
There is a bridge with lagg0 as a member for VMs that need access to multiple VLANs This bridge has an IP of 10.254.0.10 on the native (untagged) VLAN. I also see IP addresses from other VLAN interfaces when I do an ipconfig on it. These all function as expected though.
Code: 
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:fe:4a:c8:9c:00
        inet 10.254.0.10 netmask 0xffffff00 broadcast 10.254.0.255
        inet 10.11.0.50 netmask 0xffffff00 broadcast 10.11.0.255
        inet 10.12.0.52 netmask 0xffffff00 broadcast 10.12.0.255
        inet 10.11.0.51 netmask 0xffffff00 broadcast 10.11.0.255
        inet 10.20.0.50 netmask 0xffffff00 broadcast 10.20.0.255
        inet 10.12.0.51 netmask 0xffffff00 broadcast 10.12.0.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        id 00:bd:34:40:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:1a:70:8e:84:9d priority 32768 ifcost 10000 port 4
        member: tap0 flags=147<LEARNING,DISCOVER,STP,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 22 priority 128 path cost 2000000 proto rstp
                role disabled state discarding
        member: lagg0 flags=147<LEARNING,DISCOVER,STP,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 10000 proto rstp
                role root state forwarding
Jails using IP addresses on vlan11, for example, work fine:
Code: 
vlan11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=303<RXCSUM,TXCSUM,TSO4,TSO6>
        ether d0:50:99:19:2a:ab
        inet 10.11.0.50 netmask 0xffffff00 broadcast 10.11.0.255
        inet 10.11.0.51 netmask 0xffffff00 broadcast 10.11.0.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        vlan: 11 parent interface: lagg0
The problem I am having is when I attempt to create a bridge interface with a VLAN and a TAP, I am unable to pass traffic through the bridge. If I assign an IP to the bridge interface or the TAP interface I am unable to communicate with hosts on the VLAN.
Code: 
# ifconfig vlan16
vlan16: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=303<RXCSUM,TXCSUM,TSO4,TSO6>
        ether d0:50:99:19:2a:ab
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        vlan: 16 parent interface: lagg0
# ifconfig bridge7
bridge7: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:fe:4a:c8:9c:07
        inet 10.16.0.10 netmask 0xffffff00 broadcast 10.16.0.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 23 priority 128 path cost 2000000
        member: vlan16 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 55
# ifconfig tap1
tap1: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:f3:94:ee:01
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier

# ping 10.16.0.1
PING 10.16.0.1 (10.16.0.1): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- 10.16.0.1 ping statistics ---
8 packets transmitted, 0 packets received, 100.0% packet loss
A VM using the tap1 interface has no connection to the network either.

Any idea what I'm missing?

Thanks.
 
You must log in or register to reply here.
Back
Top