Is ixgbe broken on FreeBSD9 Stable?

bestwc

bestwc

I recently got a brand new Dell PowerEdge R610 server, I installed VMWare ESXi 5.0 on it and attempted to install FreeBSD as guest OS for squid caching.

The server has a PCI-E Intel X520-T2 10Gig Network Card Coper Dual Port, which is intended to be used for high speed data transfer between gateway and cache servers, but I'm now having trouble make it work under FreeBSD.

I initially install two FreeBSD boxes and added the 10gig card as e1000 to both of them, it worked but apparently with only around 1gig/s performance, which is really poor. Then I tried adding the NIC as VMxNet2 and VMxNet3, they didn't work 'cause the VMWare-Tool for FreeBSD is broken on FreeBSD 9, and the open-vm-tools compiled, installed, and didn't even detect the interface for unknown reason.

So finally I found that passthrough might be a great idea and I configured VMWare ESXi to passthrough both of the 10gig interfaces, and added them as PCI devices to my FreeBSD boxes. I thought this way the FreeBSD should detects the hardware and assign suitable driver for the NIC, but it didn't.

The problem I'm getting is I tried to re-compile kernel with ixgbe device enabled, the system detects the device correctly, assigned the driver version 2.3.3, and I was able to see ix0 interface from ifconfig. But I can't ping, can't connect to anything since the status of the device is "no carrier". I googled the error message and turns out that is a driver problem, so I downloaded the newest version of ixgbe from Intel official website, compiled and installed the driver, now the system can detect and assign driver 2.4.4 for the interface, but I'm still getting no carrier erro message.

I don't have access to the server right now, so I can't provide any output, I'll update my question once I can obtain logs.

BTW, the I believe the FreeBSD I have is the newest one, FreeBSD9-Stable, with src updated on 16 Feb 2012.

PS. I tried installing FreeBSD 9 Release version with GENERAL kernel, same problem.

Could anyone please provide me any link, thought, or troubleshoot method, thank you very much.

Regards,
Hai Lang
 
Alright, here are some outputs from the box which has the newest ixgbe driver downloaded from Intel.

Code: 
OmniSquid# dmesg | grep ix0
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.4> port 0x4000-0x401f mem 0xd2600000-0xd27fffff,0xd2400000-0xd2403fff irq 18 at device 0.0 on pci3
ix0: Using MSIX interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:d8:d7:b2
ix0: PCI Express Bus: Speed 5.0Gb/s Unknown

OmniSquid# pciconf -l -cv | grep ix
ix0@pci0:3:0:0:	class=0x020000 card=0xa03c8086 chip=0x151c8086 rev=0x01 hdr=0x00

OmniSquid# ifconfig ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	ether 00:1b:21:d8:d7:b2
	inet 10.10.10.1 netmask 0xffffff00 broadcast 10.10.10.255
	inet6 fe80::21b:21ff:fed8:d7b2%ix0 prefixlen 64 scopeid 0x4 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	[B]status: no carrier[/B]

Any idea?
 
Another interesting finding, the default driver 2.3.1 which comes with the FreeBSD 9 kernel does not seem detects media type.

Code: 
OmniSquid# dmesg | grep ix0
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.3.11> port 0x4000-0x401f mem 0xd2600000-0xd27fffff,0xd2400000-0xd2403fff irq 18 at device 0.0 on pci3
ix0: Using MSIX interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:d8:d7:b2
ix0: PCI Express Bus: Speed 5.0Gb/s Unknown

OmniSquid# ifconfig ix0 10.10.10.1 netmask 255.255.255.0 media 
ifconfig: 'media' requires argument

OmniSquid# ifconfig -m ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	capabilities=505bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,VLAN_HWFILTER,VLAN_HWTSO>
	ether 00:1b:21:d8:d7:b2
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media autoselect

But the one I downloaded from Intel can detect

Code: 
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.4> port 0x4000-0x401f mem 0xd2600000-0xd27fffff,0xd2400000-0xd2403fff irq 18 at device 0.0 on pci3
ix0: Using MSIX interrupts with 5 vectors
ix0: Ethernet address: 00:1b:21:d8:d7:b2
ix0: PCI Express Bus: Speed 5.0Gb/s Unknown

OmniSquid# ifconfig -m ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	capabilities=505bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,VLAN_HWFILTER,VLAN_HWTSO>
	ether 00:1b:21:d8:d7:b2
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media 10Gbase-T

I read from here that maybe the driver does not attempt to auto negotiate, so I tried to manually define the media type for the interface.

Code: 
OmniSquid# ifconfig ix0 10.10.10.1 netmask 255.255.255.0 media 10Gbase-T
ifconfig: SIOCSIFMEDIA (media): Invalid argument

And the dmesg displays
Code: 
ix0: Only auto media type
ix0: Only auto media type
ix0: Only auto media type

OmniSquid# ifconfig ix0 10.10.10.1 netmask 255.255.255.0 media 10Gbase-T mediaopt full-duplex
ifconfig: SIOCSIFMEDIA (media): Device not configured

OmniSquid# ifconfig ix0 10.10.10.1 netmask 255.255.255.0 media 10Gbase-T mediaopt full-duplex mtu 1500
ifconfig: SIOCSIFMEDIA (media): Device not configured

I'm pulling my hairs off, any idea?
 
Usually I use the Intel type NIC in VMWare as the 1GB is fine for me. I don't know anything about the 10GB stuff, but we usually can't help much with third party drivers, particularly since you didn't even point out the source of this driver. And, do you have open-vm-tools and if so, be sure to use version 425873,1.
 
You must log in or register to reply here.
Back
Top