em: VLAN not working any more

horst

horst

Hello,

Yesterday I've built and deployed 12.1-STABLE r361998 amd64 (GENERIC) on a
test machine and it seems I ran into a network problem (upgrade from
r359154). I didn't change the network config for approx 2 years and
upgraded every second month.

Topics involved: em, lagg, vlan

- no firewall on this machine
- routing tables look ok

None of the VLAN interfaces seem to work (inet & inet6), lagg0.600 is on
fib 0 and doesn't work neither.

However, the arp entries on the machine look ok, but not on the router
(incomplete and expired).

I can't see VLAN packets coming from this machine any more.

Any ideas?

Thanks a lot, horst

Code: 
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
        ether 0c:c4:7a:01:4d:2a
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
        ether 0c:c4:7a:01:4d:2a
        hwaddr 0c:c4:7a:01:4d:2b
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
        ether 0c:c4:7a:01:4d:2a
        inet 10.80.1.6 netmask 0xffffff00 broadcast 10.80.1.255
        inet6 fe80::ec4:7aff:fe01:4d2a%lagg0 prefixlen 64 scopeid 0x4
        inet6 2001:470:99d6:300::6:1 prefixlen 64
        laggproto loadbalance lagghash l2,l3,l4
        laggport: em0 flags=4<ACTIVE>
        laggport: em1 flags=4<ACTIVE>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0.200: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403<RXCSUM,TXCSUM,LRO>
        ether 0c:c4:7a:01:4d:2a
        inet 10.80.0.6 netmask 0xffffff00 broadcast 10.80.0.255
        inet 10.80.0.11 netmask 0xffffffff broadcast 10.80.0.11
        inet 10.80.0.12 netmask 0xffffffff broadcast 10.80.0.12
        inet 10.80.0.18 netmask 0xffffffff broadcast 10.80.0.18
        inet6 fe80::ec4:7aff:fe01:4d2a%lagg0.200 prefixlen 64 scopeid 0x5
        inet6 2001:470:99d6:200::40:1 prefixlen 64
        inet6 2001:470:99d6:200::10:1 prefixlen 64
        inet6 2001:470:99d6:200::20:1 prefixlen 64
        groups: vlan
        vlan: 200 vlanpcp: 0 parent interface: lagg0
        fib: 1
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0.400: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403<RXCSUM,TXCSUM,LRO>
        ether 0c:c4:7a:01:4d:2a
        inet 10.80.2.6 netmask 0xffffff00 broadcast 10.80.2.255
        inet 10.80.2.7 netmask 0xffffffff broadcast 10.80.2.7
        inet6 fe80::ec4:7aff:fe01:4d2a%lagg0.400 prefixlen 64 scopeid 0x6
        inet6 2001:470:99d6:400::6:1 prefixlen 64
        groups: vlan
        vlan: 400 vlanpcp: 0 parent interface: lagg0
        fib: 2
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0.600: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403<RXCSUM,TXCSUM,LRO>
        ether 0c:c4:7a:01:4d:2a
        inet 10.80.64.6 netmask 0xffffff00 broadcast 10.80.64.255
        inet6 fe80::ec4:7aff:fe01:4d2a%lagg0.600 prefixlen 64 scopeid 0x7
        inet6 2001:470:99d6:600::6:1 prefixlen 64
        groups: vlan
        vlan: 600 vlanpcp: 0 parent interface: lagg0
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 
I found the problem. Don't know why this changed but on the interfaces the option VLAN_HWTSO is missing.

Solution:

Code: 
ifconfig em0 vlanhwtso
ifconfig em1 vlanhwtso
ifconfig lagg0 vlanhwtso
 
More information: em was using LLC/ISL instead of 802.1Q Virtual LAN

Code: 
ISL
    Destination: ISL-Frame_00 (01:00:0c:00:00:00)
        0000 .... = Type: Ethernet (0)
        .... ..00 = User: Normal Priority (0)
    Source: SuperMic_01:4d:2a (0c:c4:7a:01:4d:2a) Length: 114
    DSAP: 0xaa
    SSAP: 0xaa
    Control: 0x03
    HSA: 0x0cc47a
    0000 0100 1011 000. = VLAN ID: 600
    .... .... .... ...0 = BPDU/CDP/VTP: No
    Index: 0
    Reserved: 0x0000

802.1Q Virtual LAN, PRI: 3, DEI: 0, ID: 600
    011. .... .... .... = Priority: Critical Applications (3)
    ...0 .... .... .... = DEI: Ineligible
    .... 0010 0101 1000 = ID: 600
    Type: IPv4 (0x0800)

I've found this 10 year old thread with a work-around:
https://forums.FreeBSD.org/threads/vlans-over-lagg.7668/post-65142

Currently this config works for me:

Code: 
ifconfig_em0="rxcsum txcsum -tso -lro promisc polling -vlanhwtag vlanhwtso up"
ifconfig_em1="rxcsum txcsum -tso -lro promisc polling -vlanhwtag vlanhwtso up"
 
You must log in or register to reply here.
Back
Top