Hi all,
I have a server running FreeBSD 12.2 (I know, I'll update soon...) that has been rock solid for a long time, it's now had the same thing happen twice in 2 months after years of no issues. Basically on it's primary network interface (igb0) it won't pass any traffic whatsoever.
If you try to
When it's in this state, I can still login via IPMI console and everything appears to be fine. Nothing in
The last time it happened I did save the
But then I did some maths in my head, and that seemed like junk, and sure enough, it's all junk.
Before I just blindly upgrade and hope it's fixed, has anyone else seen this? Is it likely to just be an e1000/FreeBSD kernel issue (ie, some form of memory corruption) or does it point to the BIOS? or, hope not... the hardware itself?
I have a server running FreeBSD 12.2 (I know, I'll update soon...) that has been rock solid for a long time, it's now had the same thing happen twice in 2 months after years of no issues. Basically on it's primary network interface (igb0) it won't pass any traffic whatsoever.
If you try to
ifconfig down
and then ifconfig up
the interface, it'll hang in e1000_delay
and I can't Ctrl-C it after that point.When it's in this state, I can still login via IPMI console and everything appears to be fine. Nothing in
dmesg
or /var/log/messages (other than things complaining they can't resolve addresses etc). The only thing that seems to bring it back to life is a reboot.The last time it happened I did save the
sysctl dev.igb.0
output, and noticed that crc_errs
was astonishingly high:
Code:
dev.igb.0.mac_stats.crc_errs: 3448858737885
But then I did some maths in my head, and that seemed like junk, and sure enough, it's all junk.
Code:
dev.igb.0.mac_stats.tso_ctx_fail: 3448858737885
dev.igb.0.mac_stats.tso_txd: 3449023621390
dev.igb.0.mac_stats.tx_frames_1024_1522: 3449745172500
dev.igb.0.mac_stats.tx_frames_512_1023: 3448999995106
dev.igb.0.mac_stats.tx_frames_256_511: 3448903613198
dev.igb.0.mac_stats.tx_frames_128_255: 3448913888170
dev.igb.0.mac_stats.tx_frames_65_127: 3448923975946
dev.igb.0.mac_stats.tx_frames_64: 3448858873996
dev.igb.0.mac_stats.mcast_pkts_txd: 3448858737885
dev.igb.0.mac_stats.bcast_pkts_txd: 3448858741715
dev.igb.0.mac_stats.good_pkts_txd: 3450051829491
dev.igb.0.mac_stats.total_pkts_txd: 3450051829491
dev.igb.0.mac_stats.good_octets_txd: 1439782832204
dev.igb.0.mac_stats.good_octets_recvd: 149229054078
dev.igb.0.mac_stats.rx_frames_1024_1522: 3448876895295
dev.igb.0.mac_stats.rx_frames_512_1023: 3448882897678
dev.igb.0.mac_stats.rx_frames_256_511: 3448941882198
dev.igb.0.mac_stats.rx_frames_128_255: 3449129830686
dev.igb.0.mac_stats.rx_frames_65_127: 3449180857946
dev.igb.0.mac_stats.rx_frames_64: 3448858961879
dev.igb.0.mac_stats.mcast_pkts_recvd: 3448858737885
dev.igb.0.mac_stats.bcast_pkts_recvd: 3448859286046
dev.igb.0.mac_stats.good_pkts_recvd: 3449577636257
dev.igb.0.mac_stats.total_pkts_recvd: 3449583179207
dev.igb.0.mac_stats.xoff_txd: 3448858737885
dev.igb.0.mac_stats.xoff_recvd: 3448858737885
dev.igb.0.mac_stats.xon_txd: 3448858737885
dev.igb.0.mac_stats.xon_recvd: 3448858737885
dev.igb.0.mac_stats.coll_ext_errs: 3448858737885
dev.igb.0.mac_stats.alignment_errs: 3448858737885
dev.igb.0.mac_stats.crc_errs: 3448858737885
dev.igb.0.mac_stats.recv_errs: 3448858737885
dev.igb.0.mac_stats.recv_jabber: 3448858737885
dev.igb.0.mac_stats.recv_oversize: 3448858737885
dev.igb.0.mac_stats.recv_fragmented: 3448858737885
dev.igb.0.mac_stats.recv_undersize: 3448858737885
dev.igb.0.mac_stats.recv_no_buff: 3448858737885
dev.igb.0.mac_stats.missed_packets: 3448858737885
dev.igb.0.mac_stats.defer_count: 3448858737885
dev.igb.0.mac_stats.sequence_errors: 3448858737885
dev.igb.0.mac_stats.symbol_errors: 3448858737885
dev.igb.0.mac_stats.collision_count: 3448858737885
dev.igb.0.mac_stats.late_coll: 3448858737885
dev.igb.0.mac_stats.multiple_coll: 3448858737885
dev.igb.0.mac_stats.single_coll: 3448858737885
dev.igb.0.mac_stats.excess_coll: 3448858737885
Before I just blindly upgrade and hope it's fixed, has anyone else seen this? Is it likely to just be an e1000/FreeBSD kernel issue (ie, some form of memory corruption) or does it point to the BIOS? or, hope not... the hardware itself?