re0 going down and not coming back even with netif restart

B

besta

Hello guys,

I have a frontend server running connected to a backend NFS server.

Sometimes the frontend server loses its ip addresses on re0 and is unable to renew its ip. I tried reconnecting the cables, netif restart and nothing.

This usually happens when frontend is writing data to backend NFS. Sometimes the frontend server freezes and sometimes it goes offline (maybe its frozen because of some NFS problem)

What advice could you guys give ?

I am pretty sure this is related to NFS.

frontend server /etc/fstab has:

Code: 
192.168.1.22:/mnt/nfs/_data01                                /mnt/data/_data01    nfs    rw,intr,bg,nolockd,late    0 0
192.168.1.22:/mnt/nfs/_data02                                /mnt/data/_data02    nfs    rw,intr,bg,nolockd,late    0 0

The backend NFS server has no problems.
 
Here is the log:

Code: 
Jun 24 22:34:40 e490 kernel: re0: watchdog timeout
Jun 24 22:34:40 e490 kernel: re0: link state changed to DOWN
Jun 24 22:34:43 e490 kernel: re0: link state changed to UP
Jun 24 22:34:43 e490 dhclient[56262]: New IP Address (re0): 192.168.1.3
Jun 24 22:34:43 e490 dhclient[57777]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:34:43 e490 dhclient[59136]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:34:43 e490 dhclient[61138]: New Routers (re0): 192.168.1.1
Jun 24 22:35:26 e490 kernel: re0: watchdog timeout
Jun 24 22:35:26 e490 kernel: re0: link state changed to DOWN
Jun 24 22:35:31 e490 kernel: re0: link state changed to UP
Jun 24 22:35:31 e490 dhclient[54283]: New IP Address (re0): 192.168.1.3
Jun 24 22:35:31 e490 dhclient[55901]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:35:31 e490 dhclient[56990]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:35:31 e490 dhclient[58559]: New Routers (re0): 192.168.1.1
Jun 24 22:36:16 e490 kernel: re0: watchdog timeout
Jun 24 22:36:16 e490 kernel: re0: link state changed to DOWN
Jun 24 22:36:19 e490 kernel: re0: link state changed to UP
Jun 24 22:36:19 e490 dhclient[55492]: New IP Address (re0): 192.168.1.3
Jun 24 22:36:19 e490 dhclient[56202]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:36:19 e490 dhclient[58321]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:36:19 e490 dhclient[60055]: New Routers (re0): 192.168.1.1
Jun 24 22:36:43 e490 kernel: re0: watchdog timeout
Jun 24 22:36:43 e490 kernel: re0: link state changed to DOWN
Jun 24 22:36:47 e490 kernel: re0: link state changed to UP
Jun 24 22:36:47 e490 dhclient[38914]: New IP Address (re0): 192.168.1.3
Jun 24 22:36:47 e490 dhclient[40494]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:36:47 e490 dhclient[41481]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:36:47 e490 dhclient[43868]: New Routers (re0): 192.168.1.1
Jun 24 22:37:42 e490 kernel: re0: watchdog timeout
Jun 24 22:37:42 e490 kernel: re0: link state changed to DOWN
Jun 24 22:37:45 e490 kernel: re0: link state changed to UP
Jun 24 22:37:49 e490 dhclient[60095]: New IP Address (re0): 192.168.1.3
Jun 24 22:37:49 e490 dhclient[61414]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:37:49 e490 dhclient[62313]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:37:49 e490 dhclient[63210]: New Routers (re0): 192.168.1.1
Jun 24 22:38:27 e490 kernel: re0: watchdog timeout
Jun 24 22:38:27 e490 kernel: re0: link state changed to DOWN
Jun 24 22:38:30 e490 kernel: re0: link state changed to UP
Jun 24 22:38:36 e490 dhclient[20487]: New IP Address (re0): 192.168.1.3
Jun 24 22:38:36 e490 dhclient[21255]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:38:36 e490 dhclient[22831]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:38:36 e490 dhclient[23601]: New Routers (re0): 192.168.1.1
Jun 24 22:39:20 e490 kernel: re0: watchdog timeout
Jun 24 22:39:20 e490 kernel: re0: link state changed to DOWN
Jun 24 22:39:24 e490 kernel: re0: link state changed to UP
Jun 24 22:39:24 e490 dhclient[84029]: New IP Address (re0): 192.168.1.3
Jun 24 22:39:24 e490 dhclient[85334]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:39:24 e490 dhclient[86479]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:39:24 e490 dhclient[88349]: New Routers (re0): 192.168.1.1
Jun 24 22:40:14 e490 kernel: re0: watchdog timeout
Jun 24 22:40:14 e490 kernel: re0: link state changed to DOWN
Jun 24 22:40:18 e490 kernel: re0: link state changed to UP
Jun 24 22:40:23 e490 kernel: re0: watchdog timeout
Jun 24 22:40:23 e490 kernel: re0: link state changed to DOWN
Jun 24 22:40:26 e490 kernel: re0: link state changed to UP
Jun 24 22:40:31 e490 kernel: re0: watchdog timeout
Jun 24 22:40:31 e490 kernel: re0: link state changed to DOWN
Jun 24 22:40:35 e490 kernel: re0: link state changed to UP
Jun 24 22:41:29 e490 kernel: n<f6>nfs ss serervernfs sennnfnfs server fs n192.168.1.22:/mserver n192.168.1.22:/mnt/nfst/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: /
Jun 24 22:41:29 e490 kernel: _data02: snot respondin g
Jun 24 22:41:29 e490 kernel: s
Jun 24 22:41:29 e490 kernel: erver 192. nfs 192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: f
Jun 24 22:41:29 e490 kernel: s server 192.1snfs server 192.1erver 192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: 6
Jun 24 22:41:29 e490 kernel: 8.nfs server 192.168.1.22:/mnt/nfs/_data01.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: 0
Jun 24 22:41:29 e490 kernel: 2: ver no192.168nfs .1s5.2:/mnt/nfs/_data02:er venro 192.1t r68.1.22e:spondi/mng
Jun 24 22:41:29 e490 kernel: n
Jun 24 22:41:29 e490 kernel: t/nfs/_data02: not nfsrespond singe
Jun 24 22:41:29 e490 kernel: r
Jun 24 22:41:29 e490 kernel: ver 192.168.1.22:/mnt/nfs/t respondingnfs _server 192A.168nfs server 192.168.1.22:/mnt/nfs/_d.1.22ata02: not responding
Jun 24 22:41:29 e490 kernel: nfs server 192.168.1.22:/m68.nt/n1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: :
Jun 24 22:41:29 e490 kernel: /mnt/nfs/_data02: not renfs server
Jun 24 22:41:29 e490 kernel: 1
Jun 24 22:41:29 e490 kernel: 92.168.1.22:/mnt/nfs/_data02: nfs server not respon192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: s
Jun 24 22:41:29 e490 kernel: ponding
Jun 24 22:41:29 e490 kernel: d
Jun 24 22:41:29 e490 kernel: ing
Jun 24 22:41:29 e490 kernel: f
Jun 24 22:41:29 e490 kernel: s server 192.168.1.22:/mnt/nfs/_datfa02: not respondings/_dat
Jun 24 22:41:29 e490 kernel: a
Jun 24 22:41:29 e490 kernel: 02: not responding
Jun 24 22:41:29 e490 kernel: nfs server 192nfs se.rver 1168.921.168.1.5.22:/:/mnt/nmntfs/_data02: nfs server 192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: /
Jun 24 22:41:29 e490 kernel: nfs/_data02: nnotot  responding
Jun 24 22:41:29 e490 kernel: r
Jun 24 22:41:29 e490 kernel: esponding
Jun 24 22:41:29 e490 kernel: n
Jun 24 22:41:29 e490 kernel: fs server 192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: d
Jun 24 22:41:29 e490 kernel: ata02: not responding
Jun 24 22:41:29 e490 kernel: r
Jun 24 22:41:29 e490 kernel: ver 192.168.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:29 e490 kernel: 1
Jun 24 22:41:29 e490 kernel: 68.1.22:/mnt/nfs/_data02: not responding
Jun 24 22:41:49 e490 dhclient[32139]: New IP Address (re0): 192.168.1.3
Jun 24 22:41:49 e490 dhclient[32815]: New Subnet Mask (re0): 255.255.255.0
Jun 24 22:41:49 e490 dhclient[34169]: New Broadcast Address (re0): 192.168.1.255
Jun 24 22:41:49 e490 dhclient[34975]: New Routers (re0): 192.168.1.1
Jun 24 22:41:50 e490 dhclient[36751]: New Routers (re0): 192.168.1.1
Jun 24 22:41:54 e490 kernel: re0: watchdog timeout
Jun 24 22:41:54 e490 kernel: re0: link state changed to DOWN
Jun 24 22:41:58 e490 kernel: re0: link state changed to UP
Jun 24 22:42:04 e490 kernel: re0: watchdog timeout
Jun 24 22:42:04 e490 kernel: re0: link state changed to DOWN
Jun 24 22:42:07 e490 kernel: re0: link state changed to UP
Jun 24 22:42:16 e490 kernel: re0: watchdog timeout
Jun 24 22:42:16 e490 kernel: re0: link state changed to DOWN
Jun 24 22:42:19 e490 kernel: re0: link state changed to UP
Jun 24 22:42:20 e490 kernel: nfs server 192.168.1.22:/mnt/nfs/_data01: not responding
Jun 24 22:42:24 e490 kernel: re0: watchdog timeout
Jun 24 22:42:24 e490 kernel: re0: link state changed to DOWN
Jun 24 22:42:27 e490 kernel: re0: link state changed to UP
Jun 24 22:42:36 e490 kernel: re0: watchdog timeout
...
Jun 24 23:17:16 e490 kernel: re0: link state changed to DOWN
Jun 24 23:17:19 e490 kernel: re0: link state changed to UP
Jun 24 23:17:28 e490 kernel: re0: watchdog timeout
Jun 24 23:17:28 e490 kernel: re0: link state changed to DOWN
Jun 24 23:17:30 e490 dhclient[51648]: dhclient already running, pid: 45787.
Jun 24 23:17:30 e490 dhclient[51648]: exiting.
Jun 24 23:17:31 e490 kernel: re0: link state changed to UP
Jun 24 23:17:36 e490 kernel: re0: watchdog timeout
Jun 24 23:17:36 e490 kernel: re0: link state changed to DOWN
Jun 24 23:17:39 e490 kernel: re0: link state changed to UP
Jun 24 23:17:39 e490 kernel: lo0: link state changed to DOWN
Jun 24 23:17:39 e490 dhclient[33409]: connection closed
Jun 24 23:17:39 e490 dhclient[33409]: exiting.
Jun 24 23:17:40 e490 kernel: bastille0: link state changed to DOWN
Jun 24 23:17:40 e490 kernel: lo0: link state changed to UP
Jun 24 23:17:40 e490 kernel: re0: link state changed to DOWN
Jun 24 23:17:43 e490 kernel: re0: link state changed to UP
Jun 24 23:17:52 e490 kernel: re0: watchdog timeout
Jun 24 23:17:52 e490 kernel: re0: link state changed to DOWN
Jun 24 23:17:55 e490 kernel: re0: link state changed to UP
Jun 24 23:18:00 e490 kernel: re0: watchdog timeout
Jun 24 23:18:00 e490 kernel: re0: link state changed to DOWN
Jun 24 23:18:03 e490 dhclient[88891]: connection closed
Jun 24 23:18:03 e490 kernel: lo0: link state changed to DOWN
Jun 24 23:18:03 e490 dhclient[88891]: exiting.
Jun 24 23:18:03 e490 root[70462]: /etc/rc.d/dhclient: WARNING: failed to start dhclient
Jun 24 23:18:03 e490 kernel: lo0: link state changed to UP
Jun 24 23:18:06 e490 kernel: re0: link state changed to UP
Jun 24 23:18:12 e490 kernel: re0: watchdog timeout
Jun 24 23:18:12 e490 kernel: re0: link state changed to DOWN
Jun 24 23:18:15 e490 kernel: re0: link state changed to UP
Jun 24 23:18:25 e490 dhclient[77623]: connection closed
Jun 24 23:18:25 e490 kernel: lo0: link state changed to DOWN
Jun 24 23:18:25 e490 dhclient[77623]: exiting.
Jun 24 23:18:25 e490 root[46527]: /etc/rc.d/dhclient: WARNING: failed to start dhclient
Jun 24 23:18:25 e490 kernel: re0: link state changed to DOWN
Jun 24 23:18:25 e490 kernel: lo0: link state changed to UP
Jun 24 23:18:28 e490 kernel: re0: link state changed to UP
Jun 24 23:18:35 e490 kernel: re0: watchdog timeout
Jun 24 23:18:35 e490 kernel: re0: link state changed to DOWN

Then server is rebooted
 
The re driver in base may not well work and sometimes not work at all.

Install net/realtek-re-kmod and follow the instructions concerning /boot/loader.conf.

Before to use, check that the version you installed is in sync with your kernel. The kernel version is in the package name. For example: realtek-re-kmod-1100.00.1402000_1 is designed for 14.2-RELEASE.
 
besta said:
netif restart and nothing.
Click to expand...
Restarting /etc/rc.d/netif only works for static addresses. It's /etc/rc.d/dhclient you need to restart for DHCP.

besta said:
I am pretty sure this is related to NFS.
Click to expand...
Doesn't look like it. You're getting watchdog timeouts on re0, then the interface is cycled (DOWN, then UP again). Taking the interface down causes NFS to fail, not the other way around. You need to figure out why watchdog keeps timing out.

Code: 
     re%d: watchdog timeout  The device has stopped responding to the network,
     or there is a problem with the network connection (cable).
if_re(4)
 
You must log in or register to reply here.
Back
Top