[FreeNAS] Interface stops responding after using iscsi/nfs

Network related discussions (including general TCP/IP stuff, routing, etc).

[FreeNAS] Interface stops responding after using iscsi/nfs

Postby josh4trunks » 01 May 2012, 06:00

Hey guys, I added a PCI NIC onto my FreeNAS8.2Beta3 (FreeBSD 8.2) box. I configured the onboard NIC to use DHCP, it ([FILE]re0[/FILE]) has an ip of 192.168.1.4, and I also assosiate a second IP to the onboard NIC, 192.168.1.30 (jail IP). I configured the PCI NIC to use DHCP, it ([FILE]em0[/FILE]) has an IP of 192.168.1.12. I have both NICs connected to an unmanaged switch. Everything seems to work perfectly and I can ping both NICs just fine.

The problems comes in when I use my PCI NIC for NFS or [FILE]iscsi[/FILE], I can usually connect to NFS or [FILE]iscsi[/FILE] for a few seconds / minutes on the 192.168.1.12 address but it always stops responding. At which point I can't ping 192.168.1.12 from anywhere on my network (except from being ssh'ed into the box using the 192.168.1.4 IP). I don't see anything in [FILE]syslog[/FILE] (using FreeNAS console). A local [FILE]ifconfig[/FILE] makes it seem like the interface is still working perfectly fine. I can get the interface to work again using [FILE]ifconfig em0 down[/FILE] and [FILE]ifconfig em0 up[/FILE] but using [FILE]iscsi[/FILE] or NFS makes the IP unreachable once again. [FILE]iscsi[/FILE] and NFS work perfectly ok using my 192.168.1.4 IP.

Not sure if it's the card's fault (link below), the switch's fault, or if I configured something wrongly. Thanks for the help!

http://www.newegg.com/Product/Product.aspx?Item=N82E16833106121
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby DutchDaemon » 01 May 2012, 09:54

User avatar
DutchDaemon
Old Fart
 
Posts: 10463
Joined: 16 Nov 2008, 20:17
Location: The Netherlands

Postby zennybsd » 01 May 2012, 10:57

josh4trunks wrote:Hey guys, I added a PCI NIC onto my FreeNAS8.2Beta3 (FreeBSD 8.2) box. I configured the onboard NIC to use DHCP, it ([FILE]re0[/FILE]) has an ip of 192.168.1.4, and I also assosiate a second IP to the onboard NIC, 192.168.1.30 (jail IP). I configured the PCI NIC to use DHCP, it ([FILE]em0[/FILE]) has an IP of 192.168.1.12. I have both NICs connected to an unmanaged switch. Everything seems to work perfectly and I can ping both NICs just fine.

The problems comes in when I use my PCI NIC for NFS or [FILE]iscsi[/FILE], I can usually connect to NFS or [FILE]iscsi[/FILE] for a few seconds / minutes on the 192.168.1.12 address but it always stops responding. At which point I can't ping 192.168.1.12 from anywhere on my network (except from being ssh'ed into the box using the 192.168.1.4 IP). I don't see anything in [FILE]syslog[/FILE] (using FreeNAS console). A local [FILE]ifconfig[/FILE] makes it seem like the interface is still working perfectly fine. I can get the interface to work again using [FILE]ifconfig em0 down[/FILE] and [FILE]ifconfig em0 up[/FILE] but using [FILE]iscsi[/FILE] or NFS makes the IP unreachable once again. [FILE]iscsi[/FILE] and NFS work perfectly ok using my 192.168.1.4 IP.

Not sure if it's the card's fault (link below), the switch's fault, or if I configured something wrongly. Thanks for the help!

http://www.newegg.com/Product/Product.aspx?Item=N82E16833106121



It is not clear which version of FreeNAS you are using and how jail is executed whether you are using pf/ipfw or anything. Maybe you need to compile VIMAGE for epair, just guess in the kernel. This may also require some ipfw adjustments, I guess.

What you can do is you can pop in any livecd (I suggest any lightweight linux like puppy) with the WAN connected to the new card and try to figure out if that is working (the novice way).

Else, you can post your NIC card status here in command line.
zennybsd
Member
 
Posts: 159
Joined: 14 Mar 2011, 14:07

Postby josh4trunks » 01 May 2012, 11:48

-Here's a link if the related thread I made in the FreeNAS forum, I just assumed it was OS agnostic enough to ask here / you guys might have better ability to help me debug it.
http://forums.freenas.org/showthread.php?7006-Second-NIC-keeps-locking-up&p=27245#post27245
  • FreeNAS version 8.2Beta3 (forgot to say 64bit), based on FreeBSD 8.2.
  • I believe jail is executed with [FILE]service jail start[/FILE] (freenas doesn't do everything in [FILE]init[/FILE]), I'll test with the jail off and confirm if PCI interface still hangs. (Jail is an alias of the onboard interface, the one I'm not having troubles with).
  • Neither pf or ipfw is running
  • tell me if you wanted more info
    Code: Select all
    $ifconfig em0
    em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
       ether 00:0e:04:b7:8c:e0
       inet 192.168.1.12 netmask 0xffffff00 broadcast 192.168.1.255
       media: Ethernet autoselect (1000baseT <full-duplex>)
       status: active
I'll also try running with only the PCI interface and see if this works as expected, Thanks.
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby josh4trunks » 01 May 2012, 13:34

I already replied earlier but it needs to be approved.

Code: Select all
$pciconf -lv
em0@pci0:6:1:0:   class=0x020000 card=0x13768086 chip=0x107c8086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Gigabit Ethernet Controller (Copper) rev 5 (82541PI)'
    class      = network
    subclass   = ethernet


I deleted the configuration of my onboard ([FILE]re0[/FILE]) nic, told my DHCP server to give out 192.168.1.4 to my PCI NIC ([FILE]em0[/FILE]). Unplugged the ethernet from my onboard NIC. Rebooted.

Once again everything works as expected, my PCI NIC is assigned 192.168.1.4, I can ping it. But if I use the IP for anything (including SSH) it suddenly becomes no longer reachable. Have to plug a monitor/keyboard in to restart the interface. Using the monitor/keyboard I can't even ping my router from my box once the IP stops responding.

This leads me to assume it's not something I configured weird, but something wrong with this NIC and my version of FreeNAS (based on FreeBSD8.2). This conclusion doesn't seem probable to me because it's a widely used Intel NIC but I don't know what else it could be. Any ways to debug this thing would be greatly appreciated, thanks.
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby josh4trunks » 01 May 2012, 14:18

I found someone with the same chip and same issue (Nick Rogers) http://lists.freebsd.org/pipermail/freebsd-stable/2010-January/054451.html
He also continues to ask for help in this thread though neither seem to have a resolution
http://freebsd.1045724.n5.nabble.com/em-4-ALTQ-broken-td3934794.html#a3934807
I turned off TSO and TXCSUM as they suggested in the first thread but it doesn't change the result.

Interestingly they claim it worked in FreeBSD 7.2, my goal is to have this work in FreeBSD 8.3 (kernel FreeNAS will soon be based on).
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby jaccoh » 14 May 2012, 07:16

I have the exact same issues. Using NFS in my case, locks up the interface. The only thing to do is an [FILE]ifconfig down[/FILE] and [FILE]ifconfig up[/FILE]. However, I believe it is not related to just one driver or one NIC. I tried both Realtek ([FILE]re[/FILE]) and Intel ([FILE]em[/FILE]). Both have the same problem. The solution for me is to lower the [FILE]rsize[/FILE]/[FILE]wsize[/FILE] but that gives me crappy performance.

See my post here:
http://forums.freenas.org/showthread.php?7111-NIC-keeps-locking-up
jaccoh
Junior Member
 
Posts: 3
Joined: 14 May 2012, 05:17

Postby josh4trunks » 15 May 2012, 03:14

Mind giving the output of [FILE]pciconf -lv[/FILE]? I just want to check to see if you have the same chip as I. I believe it is only the chip I have, and possibly one other, that have the hanging problem.
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby jaccoh » 15 May 2012, 07:16

jaccoh
Junior Member
 
Posts: 3
Joined: 14 May 2012, 05:17

Postby josh4trunks » 16 May 2012, 01:03

Well, you have a different chip than me. It's weird you're getting similar behavior for different types of interfaces. Not sure what to do from here, I heard the Intel driver from FreeBSD 7 should work so I may compile that and test it one of these days.
josh4trunks
Junior Member
 
Posts: 16
Joined: 04 Oct 2011, 03:23

Postby jaccoh » 16 May 2012, 05:42

I believe a set of tunables and sysctls has 'fixed' this. Or perhaps it's acting like a work-around. Nonetheless, I do not experience this problem currently. I will post the list of tunables and sysctls that I use later today. As soon as I can figure out where these are stored in file on FreeBSD/FreeNAS.
jaccoh
Junior Member
 
Posts: 3
Joined: 14 May 2012, 05:17

what was the solution

Postby cmbaker82 » 12 Jul 2012, 03:27

Could you post the settings that worked
cmbaker82
Junior Member
 
Posts: 5
Joined: 12 Jul 2012, 03:25

Postby cmbaker82 » 12 Jul 2012, 21:02

Below is what fixed the issue for me on freenas 8.2 beta4 with intel NIC:
this is from http://forums.freenas.org/showthread.php?1171-Intel-82576-Jumbo-frames-issue-(Supermicro-X8DTH-6F)-with-fix

login as root and type enter the following commands:

mount -uw /
echo kern.ipc.nmbclusters=\"131072\" >> /boot/loader.conf
echo hw.igb.num_queues=\"4\" >> /boot/loader.conf
sync ; reboot
cmbaker82
Junior Member
 
Posts: 5
Joined: 12 Jul 2012, 03:25


Return to Networking

Who is online

Users browsing this forum: apt and 1 guest