Solved Intel igc - link state changed to DOWN, UP, DOWN, UP ...

B

bsdperson

Hello!
Freebsd 14.3-RELEASE-p2

igc0 is reporting the following. It's connected to a Ubiquity USW Flex 2.5G 5 port, which shows no errors as far as I can see.

Can I just leave it be? Or do I need to fix anything and if so what. The only knob that I can possibly think to be relevant is hw.igc.max_interrupt_rate which is set to it's default of 8000.

There is also icg1 which is connected to a fiber converter. Igc1 says nothing about a change of link state.
Any ideas?

kernel log messages:
Code: 
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
+igc0: link state changed to DOWN
+igc0: link state changed to UP
 
Code: 
pciconf -lv | grep -B 4 ethernet:
igc0@pci0:2:0:0:        class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
igc1@pci0:3:0:0:        class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
 
This is not normal. Things to try:
- replace the ethernet cable
- try to connect the network interface to another switch (another brand) and see if it works properly then. Won't fix anything, but it will tell you if the problem is with the USW setup (unlikely, but worth trying).
 
Back to disabling MSI. The loader command from above is not complete. You need to add interface name.

/boot/loader.conf
hw.igc.msix_disable="1"
 
I've never had a problem with igc0 in FreeBSD or Linux.

If it was wi-fi then this behaviour can occur if there are two access points for the same network, both within range of the PC. I wonder if this could also happen in the case of ethernet switches. :-/
 
bsdperson said:
There is also icg1 which is connected to a fiber converter. Igc1 says nothing about a change of link state.
Click to expand...
I missed this part. This almost eliminates the driver as the cause of flapping? Is it possible the Ubiquity device has POE ports? I have seen that kick off link errors.
 
Phishfry said:
I missed this part. This almost eliminates the driver as the cause of flapping? I it possible the Ubiquity device has POE ports? I have seen that kick off link errors.
Click to expand...
Given it's one of those "afterwards-standardized" 2.5G thingy, my bet would also be the switch. Especially auto negotiation comes to mind...
With those in-between-standard-bandwidths (i.e. 2.5/5G) ports/switches its usually best to set them to fixed 1G link speed.

The igc driver also was pretty much hassle-free for me over the years, contrary to the em driver (e.g. [1]). However, since intel began to constantly spit out stripped/cheaped-out sub-sub-variants of their chipsets, I wouldn't rule out the driver entirely (or better say, the firmware which is incompatible with the rest of the chipset family and hence the driver). But first bet here is definitely the switch, especially if another igc interface (with the same i226 chipset!!) is working fine.



[1] https://forums.freebsd.org/threads/intel-i219-v-adl-16-unusable-with-mtu-5360.88258/
 
sko said:
The igc driver also was pretty much hassle-free for me over the years, contrary to the em driver (
Click to expand...
I had exactly the opposite experience. Up until iflib switchover EM interfaces were the best and I sneared at Realtek. Then things went sideways some.
The Intel ethernet experience went slightly rocky.

I recently got a Asrock Industrial NUC board and igc barfs while Realtek steals the show. Topsy turvy world. I have yet to investigate.
 
Phishfry said:
I missed this part. This almost eliminates the driver as the cause of flapping? Is it possible the Ubiquity device has POE ports? I have seen that kick off link errors.
Click to expand...
Yes the switch has poe+ on all ports.
Speed was set to auto negotiate on the specific port on the switch which is not so good. I missed that when faffing about with the changing of switches. Auto does not work so great on this Ubiquity gear. So now it set to a fixed 2.5. Lets see how that goes.
I had one other theory also and that was that it had to do with the amount of traffic but that proved to be a red herring.
 
sko said:
With those in-between-standard-bandwidths (i.e. 2.5/5G) ports/switches its usually best to set them to fixed 1G link speed.
Click to expand...
I agree with this. It is a 5 port pocket router. USB-C for power. Set ports to 1G.
Turn off POE if possible for that router port..
 
bsdperson said:
igc0: EEPROM V2.17-0 eTrack 0x80000308
igc1: EEPROM V2.17-0 eTrack 0x80000308

So would a firmware upgrade be a good idea?
Click to expand...
From what I can tell from the bug report, V2.17 firmware is problematic and NVM 2.25 or 2.27 may be needed.
Unfortunately, I can't check if a newer firmware version solves the issue because Intel does not provide a newer firmware version for download and I don't have an updated BIOS for my mini-pc.
If you have an update for your board, it might be worth checking.
 
Thanks all! I'll have a go at upgrading the firmware if possible.

In the meantime, current status:

- I did away with auto negotiation and set it to 2.5Gbps last might
- I ran iperf3 and got 2.38Gbps consistently over many (15+) runs
- So far no more link drops
 
bsdperson said:
It's connected to a Ubiquity USW Flex 2.5G 5 port,
Click to expand...
If it matches this: https://openwrt.org/toh/hwdata/ubiquiti/ubiquiti_usw-flex

It has a MediaTek MT7621AT; the NETGEAR R6260 I use also has that, and I noticed some devices sometimes do that up/down thing frequently if they hotplug. Not sure if it was certain cables or driver-related on the devices themselves, but I put everything on a switch then connected it to the router (the switch doesn't restart/hotplug so no instability on the connection to the router; devices on switch could then hotplug fine)
 
Espionage724 said:
If it matches this: https://openwrt.org/toh/hwdata/ubiquiti/ubiquiti_usw-flex

It has a MediaTek MT7621AT; the NETGEAR R6260 I use also has that, and I noticed some devices sometimes do that up/down thing frequently if they hotplug. Not sure if it was certain cables or driver-related on the devices themselves, but I put everything on a switch then connected it to the router (the switch doesn't restart/hotplug so no instability on the connection to the router; devices on switch could then hotplug fine)
Click to expand...
Almost, it's this specific device: https://store.ui.com/us/en/category/switching-utility/products/usw-flex-2-5g-5
 
bsdperson said:
I did set a fixed speed and just like that not a single flap.
Click to expand...
Interesting. I wonder if that implies marginal hardware somewhere, perhaps power, maybe a capacitor or something a little too big or too small. The marginal hardware is often exposed by cables which is why people will say "try a different cable". Since igc1 never had an issue I would have tried swapping the cables between igc0 and igc1 just to see what happens.
 
You must log in or register to reply here.
Back
Top