bge driver issues

Z

zdannar

I believe that I am having issues with the bge driver. The card is seen by the kernel but will not communicate. Below are the details. I did notice the IRQ being different in dmesg vs vmstat, but seen other posts that say this condition has happed before. It still seems fishy to me.

If I am missing any information please let me know.

Thanks in advance.

Code: 
[ pciconf -lv ]
bge0@pci0:2:0:0:        class=0x020000 card=0x01ec1028 chip=0x167a14e4 rev=0x02 hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'Broadcom NetXtreme Gigabit Ethernet Controller (BCM5754)'
    class      = network
    subclass   = ethernet

[ dmesg ]
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x00b002> mem 0xfd8f0000-0xfd8fffff irq 16 at device 0.0 on pci2
miibus0: <MII bus> on bge0
brgphy0: <BCM5787 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge0: Ethernet address: 00:18:8b:8b:97:2d
bge0: [FILTER]

[ vmstat -ia ]
irq256: bge0                           0          0

[ ifconfig ]
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:18:8b:8b:97:2d
        inet6 fe80::218:8bff:fe8b:972d%bge0 prefixlen 64 scopeid 0x1
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
 
I am in the wrong category in the forum? Does anybody have any ideas or need more information? Thanks.
 
Post your /etc/rc.conf, and look at a [cmd=]dmesg -a[/cmd] for additional detail on what happens exactly when networking fires up.
 
DutchDaemon,

I have included the info that you requested. I did edit the `dmesg -a` just for size purposes. I can see about attaching it if it is not enough. Notice that now that the system is up, there is a carrier but during dmesg there was not.

Thanks.

Code: 
[ RC.CONF ]
background_dhclient="YES" 
compat5x_enable="YES" 
sshd_enable="YES" 
devd_enable="YES" 
devfs_system_ruleset="devfsrules_common" 
ldconfig_paths="/usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/compat/pkg"

# Disable Sendmail by default
sendmail_enable="NONE"

# Enable console mouse
moused_type="auto" 
moused_enable="YES" 

# Enable the pcbsd startup / shutdown scripts
pcbsdinit_enable="YES"
 
#Enable samba server 
samba_enable="YES" 
winbindd_enable="YES"
 
# Disable LPD  
lpd_enable="NO" 
 
# Enable CUPS 
cupsd_enable="YES" 
linux_enable="YES"
 
# FSCK Enhancements 
fsck_y_enable="YES" 
 
# Denyhosts Startup 
denyhosts_enable="YES" 

# powerd: adaptive speed while on AC power, adaptive while on battery power
powerd_enable="YES"
powerd_flags="-a hiadaptive -b adaptive" # set CPU frequency

# enable HAL / DBUS
dbus_enable="YES"
hald_enable="YES"

# Enables support for HPLIP
hpiod_enable="NO"
hpssd_enable="NO"

# Enable the firewall
pf_rules="/etc/pf.conf"
pf_enable="YES"
pf_flags=""

# Enable sound-support
snddetect_enable="YES"
mixer_enable="YES"

# Enable avahi_daemon
avahi_daemon_enable="YES"

# Run the port jail
portjail_enable="YES"

# Added for sound support in the portjail, access to /dev/random, /dev/null, etc.
jail_pcbsd_devfs_enable="YES"

# Start the swapmonitor
swapmonitor_enable="YES"

# Enable IPV6 support
ipv6_enable="YES"

# Enable BSDStats
bsdstats_enable="YES"

# Enable webcamd
webcamd_enable="YES"

# Disable writing syslogs to remote devices by default
syslogd_flags="-ss"
warden_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
keymap="us.iso"
# Auto-Enabled NICs from pc-sysinstall
ifconfig_bge0="DHCP"
ifconfig_ue0="DHCP"
hostname="pcbsd-1472"
if_bge_load="YES"

[ dmesg -a # partial ]
Code: 
Timecounter "HPET" frequency 25000000 Hz quality 900
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.0 (no driver attached)
pci0: <memory, RAM> at device 0.1 (no driver attached)
pci0: <memory, RAM> at device 0.2 (no driver attached)
pci0: <memory, RAM> at device 0.3 (no driver attached)
pci0: <memory, RAM> at device 0.4 (no driver attached)
pci0: <memory, RAM> at device 0.5 (no driver attached)
pci0: <memory, RAM> at device 0.6 (no driver attached)
pci0: <memory, RAM> at device 0.7 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib2
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x00b002> mem 0xfd8f0000-0xfd8fffff irq 16 at device 0.0 on pci2
miibus0: <MII bus> on bge0
brgphy0: <BCM5787 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge0: Ethernet address: 00:18:8b:8b:97:2d
bge0: [FILTER]
pcib3: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci3: <ACPI PCI bus> on pcib3
vgapci0: <VGA-compatible display> port 0xbc00-0xbcff mem 0xd0000000-0xdfffffff,0xfddf0000-0xfddfffff irq 16 at device 0.0 on pci3
vgapci1: <VGA-compatible display> mem 0xfdde0000-0xfddeffff at device 0.1 on pci3
pci0: <memory, RAM> at device 9.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 10.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 10.1 (no driver attached)
pci0: <memory, RAM> at device 10.2 (no driver attached)
ohci0: <OHCI (generic) USB controller> mem 0xfe02f000-0xfe02ffff irq 21 at device 11.0 on pci0
ohci0: [ITHREAD]
usbus0: <OHCI (generic) USB controller> on ohci0
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfe02e000-0xfe02e0ff irq 22 at device 11.1 on pci0
ehci0: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <EHCI (generic) USB 2.0 controller> on ehci0
atapci0: <nVidia nForce MCP51 SATA300 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xe000-0xe00f mem 0xfe02d000-0xfe02dfff irq 23 at device 14.0 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0

....

Starting file system checks:
/dev/label/rootfs0: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/label/rootfs0: clean, 771324 free (2820 frags, 96063 blocks, 0.3% fragmentation)
/dev/label/var0: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/label/var0: clean, 474024 free (856 frags, 59146 blocks, 0.2% fragmentation)
/dev/label/usr0: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/label/usr0: clean, 107192200 free (67664 frags, 13390567 blocks, 0.1% fragmentation)
Mounting local file systems:
WARNING: TMPFS is considered to be a highly experimental feature in FreeBSD.
.
Setting hostname: pcbsd-1472
.
Starting Network: lo0 bge0 ue0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet 127.0.0.1 netmask 0xff000000 
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:18:8b:8b:97:2d
        inet6 fe80::218:8bff:fe8b:972d%bge0 prefixlen 64 tentative scopeid 0x1 
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (none)
        status: no carrier
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:25:9c:2a:44:0f
        inet6 fe80::225:9cff:fe2a:440fà prefixlen 64 tentative scopeid 0x6 
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
Enabling pf
no IP address found for lo1:network
/etc/pf.conf:4: 
could not parse host specification

no IP address found for lo1:network
/etc/pf.conf:5: 
could not parse host specification

no IP address found for lo1:network
/etc/pf.conf:6: 
could not parse host specification

pfctl: 
Syntax error in config file: pf rules not loaded

pf enabled
.
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
net.inet6.ip6.forwarding: 
0
 -> 
0

plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
 
Now we know you're using PC-BSD, but still not what version of FreeBSD. Is it 7.x, 8.x? amd64, i386?
Here's my bge0 output from 8-stable. It's a different PHY, though.
Code: 
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x003003> mem 0xfc500000-0xfc50ffff irq 20 at device 2.0 on pci5
bge0: CHIP ID 0x00003003; ASIC REV 0x03; CHIP REV 0x30; PCI
miibus0: <MII bus> on bge0
brgphy0: <BCM5705 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Ethernet address: 00:11:0a:3e:6d:0e
bge0: [ITHREAD]
 
Code: 
uname -a
FreeBSD pcbsd-1472 8.1-RELEASE FreeBSD 8.1-RELEASE #1: Sat Jul 17 14:19:59 PDT 2010     
root@build8x64.pcbsd.org:/usr/obj/usr/pcbsd-build81/fbsd-source/8.1/sys/PCBSD  amd64
 
As stated in an earlier post, one thing that confuses me is the output of the [cmd=]vmstat -ia[/cmd] where it states that the bge0 card is on irq256 yet dmesg says that it should be on irq16. I wouldn't believe that this is normal, does anybody know (Or send me to an article)

thanks.
 
Ok, well I found a post that suggested that this may be MSI which uses interrupt numbers over 256.
 
Hi,

Code: 
        media: Ethernet autoselect (none)
        [color="Red"]status: no carrier[/color]


This is interesting - are you 100% sure you have no cable issues ? Bad link, etc. If there is a problem with a driver, I would expect to see some complains in dmesg/messages and pciconf.

No carrier means there is a L-1 problem on the link.
 
Yes,

I am working with another who is having the same problems with the bge0 card. Both hosts are Dell 740 Optiplex's with BCM7454/BCM75857 cards(Different Network Connections). I listed based on the message from dmesg is different than what was listed from pciconf. See first post.

In addition,

Here is the output from an ifconfig. Notice that it claims to be active.

Code: 
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:18:8b:8b:97:2d
        inet6 fe80::218:8bff:fe8b:972d%bge0 prefixlen 64 scopeid 0x1 
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

I have tried setting the media to other settings without luck as well.

Thanks.




Thanks
 
zdannar said:
Here is the output from an ifconfig. Notice that it claims to be active.

Code: 
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:18:8b:8b:97:2d
        inet6 fe80::218:8bff:fe8b:972d%bge0 prefixlen 64 scopeid 0x1 
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
Click to expand...

That looks normal. What happens when you run dhclient(8) on it?
# dhclient bge0
 
There had been a lot of improvements in bge(4) so it would be great if you can try it on 8.2-RELEASE. If you still see the same bge(4) issue on 8.2-RELEASE, please open a new PR and let me know the PR number.
 
wblock,

Here is the output from a dhclient call. With that said, I ran the same thing yesterday(and a few days before) and had not received the send_packet error. This may be a nice hint.

Code: 
[root@pcbsd-1472] ~# dhclient bge0
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 7
send_packet: No buffer space available
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 19
send_packet: No buffer space available
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 15
send_packet: No buffer space available

yongari,

I had tried a pcbsd-8.2 (FreeBSD 8.2) live disk yesterday to verify that updates to the drivers may have fixed this issue, no luck. I will run through the process again today and verify once more. Otherwise I will open a PR. Please note the error that I had not seen before related to the send_packet error.

Code: 
[root@pcbsd-1472] ~# dhclient bge0
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 4
send_packet: No buffer space available
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 8
send_packet: No buffer space available
DHCPDISCOVER on bge0 to 255.255.255.255 port 67 interval 10
send_packet: No buffer space available

Thanks.
 
bge(4) is smart enough not to send packets until it successfully establishes a connection with link partner. There are two possible cause of seeing ENOBUFS(No buffer space available) error in user land. If bge(4) failed to establish a link you can see the ENOBUFS error. The other cause can come from dhclient since it still does not know whether it successfully established a link or seen a link UP event. Link UP event does not necessarily mean link establishment since auto-negotiation takes time to resolve.
Anyway, file a PR and let me know the PR number.
 
yongari@ said:
Link UP event does not necessarily mean link establishment since auto-negotiation takes time to resolve.
Click to expand...
Indeed. It might be interesting to know what is on the other end of the cable. On some FreeBSD boxes I have which use igb interfaces, I could not get link to come up when connecting them to a Cisco Catalyst 3750 which had some generic copper SFP's installed. The Cisco end never even displayed a link light, while the link lights would cycle on and off on the FreeBSD systems, even with the system in standby (so not a driver problem). Swapping the SFP's out for some SGS-branded ones fixed the problem. It took me some time to isolate it to the SFP's, as anything else I plugged into them worked. Of course, the vendor said "we've never heard of that happening with anyone else", but did refund my money.

As you know (but I'll repeat for our at-home audience), modern network adapter chips are complete computer systems, and occasionally need firmware updates. That is the internal firmware that the adapter chip uses, not the boot ROM, etc. Both Intel and Broadcom release firmware updates for their chips, though you probably have to go to your system integrator / motherboard manufacturer to get it.

yongari, would it be useful to have the user(s) experiencing this problem run the Broadcom b57diag from a DOS boot disk and tell you what firmware their cards are running?
 
yongari,

I have submitted the PR and will post the number as soon as it's available.

Terry,

I will get back to you question/solution just a bit later.
 
Terry,

The direct cable attached to the host is connected to a linksys SD2005 switch. From there, I am not sure what the upward components are. With that said, there are cisco catalyst switches on site. It may very well be under a 3750. The immediate switch on my desk(SD2005) show positive links from on the switch and the NIC. I have also tried connecting other active cables in the NIC and performing the same tests as a friend in a different part of the building with the same problem.

Regarding the firmware, I would be willing to give it a look. The problem host was just a run of the mill Dell optiplex 740 that was taking up space in the next room. I will look into the firmware updates. It will be interesting to see what yongari says.

Once again, thanks for all of the help from everybody.
 
It's possible for vendor to release several firmware releases for smart controllers like bge(4) and em(4)/igb(4). These firmwares are mainly for additional features like ASF/IPMI and PXE. However it's possible to make the firmware code to load DSP fixups which may address link establishment issues. I guess vendor's release note for the firmware may have an entry for link establishment improvement if they fixed link establishment issues.

I don't know what were fixed in each firmware release so knowing exact firmware version wouldn't help. To me, controller model, dmesg output and full description of issue is better to narrow down the issue.
 
You must log in or register to reply here.
Back
Top