Way to get alert on duplicate bridge IP address?

H

hpnothp

I accidentally created 2 bridges with same IP/mask (forgot to delete old one still active from /etc/rc.conf and created duplicate with vm-bhyve commands).

Question: Is there some sysctl to enable warnings from kernel when duplicate IP address on bridge is detected?

It took me several hours (my fault) - only when I compared ARP table entries of gateway on Host and Guest - there was MAC address mismatch (each one using different bridge due same IP).
My host system: FreeBSD 15.0-PRERELEASE 1500063
Guest (running using vm-bhyve scripts): openSUSE LEAP 15.6 (but not guest related)
 
hpnothp said:
Is there some sysctl to enable warnings from kernel when duplicate IP address on bridge is detected?
Click to expand...
You'll find error/warning messages in /var/log/messages. The system is going to complain about it when they both try to send traffic traffic. You'll see messages like "A.B.C.D moved from abcd:0010:1122 to dead:beef:f00d" followed by other messages with the same IP but different MAC addresses. And it'll keep switching those MAC addresses for the same IP address.

Duplicate IPs are relatively easy to find. Duplicate MAC addresses however are a royal pain in the backside to track down and have some really weird consequences.
 
SirDice said:
You'll find error/warning messages in /var/log/messages. The system is going to complain about it when they both try to send traffic traffic. You'll see messages like "A.B.C.D moved from abcd:0010:1122 to dead:beef:f00d" followed by other messages with the same IP but different MAC addresses. And it'll keep switching those MAC addresses for the same IP address.

Duplicate IPs are relatively easy to find. Duplicate MAC addresses however are a royal pain in the backside to track down and have some really weird consequences.
Click to expand...

Unfortunately I'm unable to find such messages in my case (affected bridge uses 10.99.99.1/24 network):
Bash: 
fgrep -i ' moved from ' /var/log/messages

(no output)

fgrep  '10.99.99.1' /var/log/messages

(no output)

I found two weird things (but see "Possible explanation:" below):
  • bridge in ARP cache does not show as "ethernet" but "bridge"
  • even when I freshly ping to bridge's IP address from Host it will not appear in ARP cache - but other (unused bridge 192.168.122.1/24) is there. However communication with 10.99.99.1 bridge works without issues
Here is an example:

Bash: 
$ ping -c 1 10.99.99.1 | grep loss

1 packets transmitted, 1 packets received, 0.0% packet loss

$ arp -an
? (192.168.10.55) at f8:b1:56:9b:53:fe on em0 permanent [ethernet]  ## my hosts NIC IP address
? (192.168.10.254) at 00:90:27:fe:b8:24 on em0 expires in 1197 seconds [ethernet] ## network gateway
? (192.168.122.1) at 5e:e5:7b:73:9e:28 on vm-default permanent [bridge]    ## my other bridge (!)
? (10.99.99.33) at 58:9c:fc:00:0c:59 on vm-private expires in 602 seconds [bridge] ## IP of VM connected to 10.99.99.1/32 bridge via tap0

Possible explanation: when I manually deleted conflicting bridge, it also deleted permanent entry from ARP table, but "proper" entry was not added (because "correct" bridge configuration did not change). I will test that with reboot.

Here is full output from "ifconfig -u" for completeness:

Code: 
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=4e520bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
    ether f8:b1:56:9b:53:fe
    inet 192.168.10.55 netmask 0xffffff00 broadcast 192.168.10.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether 8e:ee:f8:15:f7:23
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=0<>
    member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            port 1 priority 128 path cost 20000 vlan protocol 802.1q
    groups: bridge vm-switch viid-4c918@
    nd6 options=9<PERFORMNUD,IFDISABLED>
vm-default: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether 5e:e5:7b:73:9e:28
    inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=0<>
    groups: bridge vm-switch viid-c21f9@
    nd6 options=9<PERFORMNUD,IFDISABLED>
vm-private: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=10<VLAN_HWTAGGING>
    ether c2:5d:39:62:02:68
    inet 10.99.99.1 netmask 0xffffff00 broadcast 10.99.99.255
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    bridge flags=0<>
    member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            port 3 priority 128 path cost 2000000 vlan protocol 802.1q
    groups: bridge vm-switch viid-2c17c@
    nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: vmnet/128-gitlab-cz-pve/0/private
    options=4080000<LINKSTATE,MEXTPG>
    ether 58:9c:fc:10:c4:87
    groups: tap vm-port
    media: Ethernet 1000baseT <full-duplex>
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    Opened by PID 24916
 
I have verified on clean system (version: 1500063) that creating bridge with duplicate IP address will not cause error and there will be no warning in /var/log/messages or dmesg output:

Here is how clean system looks - no bridge defined:
Bash: 
# ifconfig -u
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
        ether 52:54:00:20:f1:09
        inet 192.168.122.194 netmask 0xffffff00 broadcast 192.168.122.255
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

1st step - creating bridge0 with IP address:
Bash: 
# ifconfig bridge create
# ifconfig bridge0 inet 10.99.99.1/24
# ifconfig bridge0

bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=10<VLAN_HWTAGGING>
        ether 58:9c:fc:10:f2:ca
        inet 10.99.99.1 netmask 0xffffff00 broadcast 10.99.99.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        bridge flags=0<>
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

# arp -an

? (192.168.122.194) at 52:54:00:20:f1:09 on vtnet0 permanent [ethernet]
? (192.168.122.1) at 52:54:00:6f:54:be on vtnet0 expires in 1161 seconds [ethernet]
? (10.99.99.1) at 58:9c:fc:10:f2:ca on bridge0 permanent [bridge]

# ping -c 1 10.99.99.1

PING 10.99.99.1 (10.99.99.1): 56 data bytes
64 bytes from 10.99.99.1: icmp_seq=0 ttl=64 time=0.099 ms

--- 10.99.99.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.099/0.099/0.099/0.000 ms

2nd step - create new bridge with duplicate IP address:

Bash: 
# ifconfig bridge create

bridge1

# ifconfig bridge1 inet 10.99.99.1/24
# ifconfig bridge1

bridge1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=10<VLAN_HWTAGGING>
        ether 58:9c:fc:10:bd:b2
        inet 10.99.99.1 netmask 0xffffff00 broadcast 10.99.99.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        bridge flags=0<>
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

# arp -an

? (192.168.122.194) at 52:54:00:20:f1:09 on vtnet0 permanent [ethernet]
? (192.168.122.1) at 52:54:00:6f:54:be on vtnet0 expires in 1107 seconds [ethernet]
? (10.99.99.1) at 58:9c:fc:10:f2:ca on bridge0 permanent [bridge]
? (10.99.99.1) at 58:9c:fc:10:bd:b2 on bridge1 permanent [bridge]

Notice that bridge with duplicate IP address was happily created and there are even 2 ARP permanent table entries with duplicate IP address - that should at least produce warning.

Ping surprisingly still works, no error reported, ARP table still contains duplicate entries:
Bash: 
# ping 10.99.99.1

PING 10.99.99.1 (10.99.99.1): 56 data bytes
64 bytes from 10.99.99.1: icmp_seq=0 ttl=64 time=0.075 ms
64 bytes from 10.99.99.1: icmp_seq=1 ttl=64 time=0.337 ms
64 bytes from 10.99.99.1: icmp_seq=2 ttl=64 time=0.140 ms
^C
--- 10.99.99.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.075/0.184/0.337/0.112 ms

# arp -an
? (192.168.122.194) at 52:54:00:20:f1:09 on vtnet0 permanent [ethernet]
? (192.168.122.1) at 52:54:00:6f:54:be on vtnet0 expires in 1092 seconds [ethernet]
? (10.99.99.1) at 58:9c:fc:10:f2:ca on bridge0 permanent [bridge]
? (10.99.99.1) at 58:9c:fc:10:bd:b2 on bridge1 permanent [bridge]

When I delete older bridge0 ARP will contain only one permanent entry:
Bash: 
# ifconfig bridge0 destroy
# arp -an

? (192.168.122.194) at 52:54:00:20:f1:09 on vtnet0 permanent [ethernet]
? (192.168.122.1) at 52:54:00:6f:54:be on vtnet0 expires in 1028 seconds [ethernet]
? (10.99.99.1) at 58:9c:fc:10:bd:b2 on bridge1 permanent [bridge]

There is no error, no warning in dmesg or /var/log/messages, only:
Code: 
bridge0: Ethernet address: 58:9c:fc:10:f2:ca
bridge1: Ethernet address: 58:9c:fc:10:bd:b2

Summary:
  1. When I create 2nd bridge with duplicate IP address (by mistake) - there is no error reported and ARP table will contain duplicate entries
  2. There is no warning or error in /var/log/messages regarding duplicate IP address

Only (new) mystery is - why on my original machine there was no ARP entry left when I deleted 1st bridge - there should be still ARP permanent entry from 2nd bridge.
 
You must log in or register to reply here.
Back
Top