Solved Network cuts in and out. Looking for next steps.

I'm pretty stumped.

I had followed the handbook to setup a lagg interface on my Freebsd 12.1 server a few months ago and everything had been working as it should until a couple of days ago.
This is how it is and has been setup for about 4 months:
Bash:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:25:90:dc:ab:50
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:25:90:dc:ab:50
    hwaddr 00:25:90:dc:ab:51
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:25:90:dc:ab:50
    hwaddr 00:15:17:37:22:38
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:15:17:37:22:39
    inet 192.168.5.1 netmask 0xfffffffc broadcast 192.168.5.3
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:25:90:dc:ab:50
    inet 10.7.7.1 netmask 0xffffff00 broadcast 10.7.7.255
    laggproto lacp lagghash l2,l3,l4
    laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    groups: lagg
    media: Ethernet autoselect
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
This had been working very well. I was very pleased with how clear the handbook is on setting this up.

I can't really say what caused the change. The only thing I have done yesterday is port forward one port to my server.
All of a sudden it cannot stay connected to the network and I am not sure how to troubleshoot this specific issue.
Basically, what is happening is that the internet connection keeps cutting out for just a second on the server, all interfaces.
No other computers on my network have this issue.

To test, I have tried to leave an ssh session connected, which gives a
Bash:
Connection to 10.7.7.1 port 22: Broken pipe
two seconds after being connected. iperf3 from my server to my router gives very lousy performance, a full 200 Mbits less than my iMac also tested with iperf3.
When using the transmission daemon, it will give an error about disconnecting and connects again right away.
Same with plexmediaserver, samba, and with all network services. I cannot remain connected to said iMac if I login to freebsd via rs-232 and ssh in either.
The NAS on em3 is able to remain connected however.

Here is what I have done:
-Broken the trunk on the switch and removed the lag entries from my /etc/rc.conf, and assigned an IP to each of the em interfaces, which gives the same issue with a slightly longer delay before disconnecting.
-Removed the NAT rule from my firewall, closing the port. No effect.
-Unplugged the switch from my firewall and plugged the server directly into the firewall. No effect.
-Changed cables around. No effect.
-Removed pcie nic card. No effect.

I guess what I really need are some next steps. How can I tell if its a stance hardware issue or if it is certainly software related?
My switch shows nothing wrong with the server nics at all. I'll try to post as much info as needed.

Thank you for any help in advance!
 

Attachments

  • switch2.png
    switch2.png
    34.9 KB · Views: 173
  • switch1.png
    switch1.png
    18.1 KB · Views: 168
I rebuilt the server before realizing that it was the connection on my desktops that had gone bad, and the server was running fine. Sorry everyone.
 
Back
Top