arp message about an IP in daily output

Understudy

Understudy

Hi All,
So recently I had to make some changes to my network system. I put in a new router / firewall with a new IP address. The IP is within the subnet and the public gateway remains the same. However one of my servers is sending me this in the daily security run output.
Code: 
nexus.brendhanhorne.com kernel log messages:
+arp: xxx.xxx.xxx.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1
+arp: xxx.xxx.xxx.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1
+arp: xxx.xxx.xxx.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1
+arp: xxx.xxx.xxx.94 moved from 0c:c4:7a:59:7c:0f to 0c:c4:7a:59:7c:11 on bce1
+arp: xxx.xxx.xxx.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1

Server info:
FreeBSD nexus.brendhanhorne.com 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC amd64
Upgraded from 12.3 about a week ago.
The .94 IP is the router/firewall but the server has a different public IP and the gateway is a separate device upstream with it's own public IP. The router/firewall got changed about 2 weeks ago.

How do I address this message from the server?

Thank you
 
IP address conflict between two NIC and both of them claim that they have the same IP address. As the mac address is different only on the last digit this mean it's on the same hardware. You need to locate both NIC 0c:c4:7a:59:7c:11 and 0c:c4:7a:59:7c:0f and resolve the IP address conflict between them by leaving only one of the NIC with that IP address.

0c:c4:7a Super Micro Computer, Inc.
 
SirDice said:
The old gateway with that IP address still online?
Click to expand...
The gateway was and is a .81 That has been set in rc.conf before and after the changes took place. I have three servers behind the DMZ and only this one sends me this in the daily output.
 
VladiBG said:
IP address conflict between two NIC and both of them claim that they have the same IP address. As the mac address is different only on the last digit this mean it's on the same hardware. You need to locate both NIC 0c:c4:7a:59:7c:11 and 0c:c4:7a:59:7c:0f and resolve the IP address conflict between them by leaving only one of the NIC with that IP address.

0c:c4:7a Super Micro Computer, Inc.
Click to expand...



Server sending me the daily security output
Code: 
bhorne@nexus:~ $ less /etc/rc.conf
hostname="nexus.brendhanhorne.com"
ifconfig_bce1="inet xxx.xxx.xxx.89 netmask 255.255.255.240"
defaultrouter="xxx.xxx.xxx.81"
sshd_enable="YES"
moused_enable="YES"
ntpdate_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
# powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
apache24_enable="YES"
mysql_enable="yes"
mysql_pidfile="/var/db/mysql/mysql.pid"
mysql_optfile="/usr/local/etc/mysql/my.cnf"
## NFS SERVER
rpcbind_enable="YES"
nfs_server_enable="YES"
mountd_flags="-r"
mountd_enable="YES"
## NFS CLIENT
#nfs_client_enable="YES"

Server sending me the daily security output
Code: 
bhorne@nexus:~ $ ifconfig -a
bce0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether d4:85:64:7b:80:96
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether d4:85:64:7b:80:98
        inet xxx.xxx.xxx.89 netmask 0xfffffff0 broadcast xxx.xxx.xxx.95
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether d4:85:64:7b:80:9a
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bce3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether d4:85:64:7b:80:9c
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bhorne@nexus:~ $

Code: 
bhorne@nexus:~ $ arp -a
? (xxx.xxx.xxx.81) at 40:8d:5c:b0:28:24 on bce1 expires in 1200 seconds [ethernet]
? (xxx.xxx.xxx.89) at d4:85:64:7b:80:98 on bce1 permanent [ethernet]
? (xxx.xxx.xxx.94) at 0c:c4:7a:59:7c:11 on bce1 expires in 1044 seconds [ethernet]
bhorne@nexus:~ $

From the router firewall (pfsense 2.6)
Code: 
[2.6.0-RELEASE][admin@Ignis.brendhanhorne.com]/root: ifconfig
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:0c
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:0d
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:0e
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: DMZ
        options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:0f
        inet6 fe80::ec4:7aff:fe59:7c0f%ix3 prefixlen 64 scopeid 0x4
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:10
        inet6 fe80::ec4:7aff:fe59:7c10%ix4 prefixlen 64 scopeid 0x5
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:59:7c:11
        inet6 fe80::ec4:7aff:fe59:7c11%ix5 prefixlen 64 scopeid 0x6
        inet xxx.xxx.xxx.94 netmask 0xfffffff8 broadcast xxx.xxx.xxx.95
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
        groups: enc
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33160
        groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
        groups: pfsync
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: Bridge0
        ether 02:9b:09:a6:e4:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ix3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: ix5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 55
        groups: bridge
        nd6 options=1<PERFORMNUD>
[2.6.0-RELEASE][admin@Ignis.brendhanhorne.com]/root:
 
Check ix5 and ix3 and why do you need them on bridge0. Is there some kind of HA like HSRP to switch the ip address from ix5 to ix3? If it so this will explain why you see the ARP message.
 
ix3 and ix5 are bridged because ix3 is the DMZ and ix5 is the WAN.
The WAN has a public IP and and the servers behind the DMZ are public IP also
And I would love to show you the section from pfsense on public IPs behind a DMZ and it having to be bridged, except right now they are working on a DNS issue. (Thank you netgate twitter account). I offered to help (sort of)
In a nutshell the WAN has .94 and the subnet is /29 which is what the IPs on the servers fall into. As soon as I can I will post copy of the documentation.
 
VladiBG said:
Do you have any media disconnects on ix3?
Click to expand...
Just the arp messages.
Results of dmesg | less
Code: 
lo0: link state changed to UP
bce1: link state changed to UP
ums0 on uhub0
ums1 on uhub4
ums1: <vendor 0x0624 AF603A-USB2 VM Module, class 0/0, rev 2.00/1.00, addr 2> on
 usbus5
ums0: <Virtual Mouse> on usbus0
ums1: 5 buttons and [XYZ] coordinates ID=1
ums1: 5 buttons and [Z] coordinates ID=4
Security policy loaded: MAC/ntpd (mac_ntpd)
arp: 104.219.179.94 moved from 0c:c4:7a:59:7c:0f to 0c:c4:7a:59:7c:11 on bce1
arp: 104.219.179.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1
arp: 104.219.179.94 moved from 0c:c4:7a:59:7c:0f to 0c:c4:7a:59:7c:11 on bce1
arp: 104.219.179.94 moved from 0c:c4:7a:59:7c:11 to 0c:c4:7a:59:7c:0f on bce1
 
I think the problem is in the bridge. When arp request is recieved via ix3 for xxx.xxx.xxx.94 the bridge0 cause the arp reply to go out via broadcast with the mac address of the interface on which the arp request has been received in this case it's ix3 mac address which triggers the arp spoofing warning message "arp: xxx.xxx.xxx.94 moved from :11 to :0f" on all hosts connected on the network behind ix3/ix5.
As the sko already suggested if you put the IP address on the bridge instead on ix5 interface you will stop receiving those warnings.
Or if you want to just ignore them you can set the sysctl net.link.ether.inet.log_arp_movements=0
 
VladiBG said:
I think the problem is in the bridge. When arp request is recieved via ix3 for xxx.xxx.xxx.94 the bridge0 cause the arp reply to go out via broadcast with the mac address of the interface on which the arp request has been received in this case it's ix3 mac address which triggers the arp spoofing warning message "arp: xxx.xxx.xxx.94 moved from :11 to :0f" on all hosts connected on the network behind ix3/ix5.
As the sko already suggested if you put the IP address on the bridge instead on ix5 interface you will stop receiving those warnings.
Click to expand...
Yeah, some discussion on the netgate/pfsense forum is basically saying the same and possibly doing a spoof on the mac address. I may have to try that tonight.
 
Did some testing and came up with a solution, possibly a work around.
I was bothered by a couple of things. I was getting this arp report only from one of the servers behind the DMZ.
I did the changing of the Bridge to WAN and spoofed the addresses but I would still get an arp message to show up with dmesg | grep arp about every 30 minutes at the server. I tried swapping the macs and then I would just get the same message from the server with the macs being swapped. So I decided to go after the server. I tried looking into arp and tried arp -ad but it would eventually go back to the settings it had and the problem would return arp -a would show that. I ended up finding something called static arp pairs which can be put into /etc/rc.conf. Basically that pairs a mac with an IP even if it in not on that machine. So I added this to my /etc/rc.conf
Code: 
## Static Arp Pairs
static_arp_pairs="myarp"
static_arp_myarp="xxx.xxx.xxx.94 0c:c4:7a:59:7c:11"

And then ran

service static_arp start
That appears to have fixed the issue. Which I would say is on the server not the firewall. What I am not aware of is I guess arp runs in the background, maybe?
 
I appreciate that. I was vague in my response. I should have said I was wondering what what calling arp in this case to cause the output. Because it was only happening on one server. But I will take the smack. I deserved it
 
You must log in or register to reply here.
Back
Top