Replacing realtek "re" driver

J

jjlawren

I've installed 10.2 (since upgraded to 10.3) on a low-power integrated board (http://www.asrock.com/mb/Intel/N3150DC-ITX). I want to use it as a backup host, receiving ZFS snapshots from another box and storing to an external USB3 drive. However, the network has been very unstable with poor throughput and many NIC driver restarts (ie, "re0: watchdog timeout"). It has no expansion slots, so a replacement NIC is not an option.

I've read in several posts that the Realtek drivers are more stable than the built-in kernel drivers. However, I'm really not sure how to go about this. This post seems to describe what I'm trying to do, but I cannot compile the driver.

Steps I've taken:
  1. Installed 'base' under /usr/src
  2. Compiled with 'make buildworld' (which took almost a day on this hardware)
  3. Downloaded Realtek drivers and tried to compile (from outside the kernel sources)
This fails with the following errors:
Code: 
root@myhost:~/realtek/rtl_bsd_drv_v190 # make
make: "/sys/conf/kmod.mk" line 461: Could not find bsd.clang-analyze.mk
make: Fatal errors encountered -- cannot continue
make: stopped in /root/realtek/rtl_bsd_drv_v190
Do I need to update the makefile to point to the kernel sources?
 
jjlawren said:
I've read in several posts that the Realtek drivers are more stable than the built-in kernel drivers.
Click to expand...
I would like to know more about that. But then, it's news to me that Realtek even has their own version of a driver.

bsd.clang-analyze.mk appears to only be present in 11-CURRENT.
 
wblock@ said:
bsd.clang-analyze.mk appears to only be present in 11-CURRENT.
Click to expand...
Great catch. I switched to the stable/10 branch. I'm now able to build, but not load the module:
Code: 
KLD if_ree.ko: depends on kernel - not available or version mismatch
linker_load_file: Unsupported file type
Perhaps I have to use the release/10.3.0 branch explicitly? I'm not using a custom kernel.

Edit: Yep, loads just fine when using the actual release branch. Now to test!
 
I got the same problem too, but I am using pfSense 2.2(FreeBSD 10.1)/2.3(FreeBSD 10.3).

My machine is a Intel n3150 + Realtek GbE too. The problem can be tested by running iperf tests.
http://www.hkepc.com/forum/viewthread.php?fid=12&tid=2285374&page=1&extra=#pid34784312
http://www.hkepc.com/forum/viewthread.php?fid=12&tid=2240827

I found that locking the adaptor to 100M can avoid packet loss and disconnections, however this shouldn't be a solution.

My temporary solution is to get a PCIe 1x to mini-PCIe adaptor, and install an Intel GbE card.
 
I got the official Realtek GbE driver for FreeBSD 9.x complied on FreeBSD 10.3, and installed it on pfSense 2.3 (FreeBSD 10.3).

The disconnection problem is now gone, but the performance still bad. Will try other tunables.

dmesg
pfSense_realtek_re.png


iperf3 result
pfSense_realtek_v19_iperf.png
 
I have Intel N3050 based board with the following RealTek chipset:

Code: 
re0@pci0:3:0:0: class=0x020000 card=0x81681849 chip=0x816810ec rev=0x11 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet

Standard FreeBSD driver times out fairly regularly. I was using RealTek provided driver v.1.88 (http://www.realtek.com.tw/downloads...d=5&Level=5&Conn=4&DownTypeID=3&GetDown=false) all the way through 9-STABLE and 10-STABLE with the pleasant results (990 Mbit/s on iperf, 110-120Mbytes/sec SMB file copy). With the upgrade to 11-STABLE driver would no longer compile, so I had to come up with the attached patch. The patch is against RealTek driver v.1.91. The system serves as the home fileserver and hosts VIMAGE jail with OpenVPN and PF to allow for outside access. So far, so good...
 

Attachments

I just ordered an Intel NUC5CPYH without knowing about the issues with the NIC driver.
The NUC has the RealTek 8111HN chip, which is not mentioned anywhere on the driver page.
The page above mentioned by Incanter mentions different RTL8111 chips, but not the 8111HN.
In the readme for 1.91 the 8111 is not mentioned at all...

What are the steps to install the RealTek provided driver with all the needed patches?
Do I just follow the instructions in the readme and add the patch by Incanter?

I wish that RealTek would just open source their driver...

Here is the deal explained for linux:
https://unixblogger.com/2016/08/11/how-to-get-your-realtek-rtl8111rtl8168-working-updated-guide/
 
Quite honestly put, get something else that uses an Intel NIC or you have an option to install for example a mini pci-express Intel NIC. I wouldn't trust Realtek to provide proper drivers for FreeBSD, FreeBSD is a moving target on the kernel programming level and while they might be able to support let's say 10.3-RELEASE there's no guarantee they will be able to provide the drivers for FreeBSD 11.
 
RealTek is now a default nic chip for a lot of motherboards, NUCs and appliances. I got n3050 motherboard with dual gigabit ports, and both of them hang after some time with "watchdog timeout" message. In case I disable virtually everything like -txcsum -tso and so on, after some time I get packet loss of 80% instead of hangs, but at least I can try to reboot the box. I even wanted to install mini pci-e gigabit adapter, but whatever could be found on aliexpress is realtek based. USB 3.0 is also out of the question, because it's a great change of hitting realtek based adapter and have the best of both worlds - usb and realtek bugs.
I'll try to compile a custom kernel for newly released freebsd 11 with realtek custom driver and patch and report results.

here is pciconf for my adapters, 10.3 stable:
Code: 
re0@pci0:1:0:0: class=0x020000 card=0xe0001458 chip=0x816810ec rev=0x0c hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet
    bar   [10] = type I/O Port, range 32, base 0xe000, size 256, enabled
    bar   [18] = type Memory, range 64, base 0x81400000, size 4096, enabled
    bar   [20] = type Prefetchable Memory, range 64, base 0xa0100000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit
    cap 10[70] = PCI-Express 2 endpoint IRQ 1 max data 128(128) RO link x1(x1)
                 speed 2.5(2.5) ASPM disabled(L0s/L1)
    cap 11[b0] = MSI-X supports 4 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x800]
    cap 03[d0] = VPD
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
    ecap 0002[140] = VC 1 max VC0
    ecap 0003[160] = Serial 1 01000000684ce000
    ecap 0018[170] = LTR 1
     Corrected = Receiver Error
re1@pci0:2:0:0: class=0x020000 card=0xe0001458 chip=0x816810ec rev=0x0c hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet
    bar   [10] = type I/O Port, range 32, base 0xd000, size 256, enabled
    bar   [18] = type Memory, range 64, base 0x81300000, size 4096, enabled
    bar   [20] = type Prefetchable Memory, range 64, base 0xa0000000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit
    cap 10[70] = PCI-Express 2 endpoint IRQ 1 max data 128(128) RO link x1(x1)
                 speed 2.5(2.5) ASPM disabled(L0s/L1)
    cap 11[b0] = MSI-X supports 4 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x800]
    cap 03[d0] = VPD
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
    ecap 0002[140] = VC 1 max VC0
    ecap 0003[160] = Serial 1 01000000684ce000
    ecap 0018[170] = LTR 1
     Corrected = Receiver Error
 
Drivers works on 11-stable. So far - so good.
How to build:
Code: 
1. Get kernel sources in /usr/src, driver 1.91 from realtek site and patch from this thread. Apply patch to driver.
2. Modify /usr/src/sys/amd64/conf/GENERIC and delete line "device re" - so if_re no longer be included in kernel itself - module will be build anyway.
3. Edit /boot/loader.conf adding line if_re_load="YES", so external module will be loaded.
4. Go to /usr/src and run "make buildkernel KERNCONF=GENERIC && make installkernel KERNCONF=GENERIC"
5. Reboot with new kernel. System should boot with open-source driver as module. kldstat will show if_re in list of modules.
6. Put patched if_re.c and if_rereg.h into /usr/src/sys/dev/re
7. Build driver (cd /usr/src/sys/modules/re && make clean && make)
8. Backup /boot/kernel/if_re.ko, then cp /usr/src/sys/modules/re/if_re.ko /boot/kernel/if_re.ko && chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko
9. Reboot

result:
Code: 
FreeBSD 11.0-PRERELEASE #25: Tue Oct 11 15:26:52 EEST 2016

re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether 40:8d:5c:aa:aa:aa
        inet 10.10.10.10 netmask 0xffffff00 broadcast 10.10.10.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

re1: <Realtek PCIe GBE Family Controller> port 0xd000-0xd0ff mem 0x81300000-0x81300fff,0xa0000000-0xa0003fff irq 17 at device 0.0 on pci2
re1: Using Memory Mapping!
re1: Using 1 MSI-X message
re1: ASPM disabled
re1: version:1.91
re1: Ethernet address: 40:8d:5c:aa:aa:aa

This product is covered by one or more of the following patents:
US6,570,884, US6,115,776, and US6,327,625.
re1: Ethernet address: 40:8d:5c:aa:aa:aa
 
Incanter said:
I have Intel N3050 based board with the following RealTek chipset:

Code: 
re0@pci0:3:0:0: class=0x020000 card=0x81681849 chip=0x816810ec rev=0x11 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet

Standard FreeBSD driver times out fairly regularly. I was using RealTek provided driver v.1.88 (http://www.realtek.com.tw/downloads...d=5&Level=5&Conn=4&DownTypeID=3&GetDown=false) all the way through 9-STABLE and 10-STABLE with the pleasant results (990 Mbit/s on iperf, 110-120Mbytes/sec SMB file copy). With the upgrade to 11-STABLE driver would no longer compile, so I had to come up with the attached patch. The patch is against RealTek driver v.1.91. The system serves as the home fileserver and hosts VIMAGE jail with OpenVPN and PF to allow for outside access. So far, so good...
Click to expand...
Hey Incanter, do you happen to have that kernel module and would you mind uploading to google drive or elsewhere? I say Kenny's version above but I'm guessing that is v1.92 which he was having issues with throughput. I'd like to try the 1.88 version but the Realtek site no longer has it from what I can see.
 
Hi, I came across this post while researching for the same issue. My Realtek NIC (RTL8111E) does not play well with FreeNAS 9.10-stable (which built on FreeBSD 10.3), keep auto negotiating at 100mb instead of Gigabit. I see that Realtek has released a new driver ver 1.92 on 10/25/2016, package includes the "if_re.c" and "if_rereg.h" files. From what I read I need to compile the ".c" file into a ".ko" but I'm new to Linux and I'm still a little confused about compiling.

If anyone have a copy of the compiled ".ko" with the newest Realtek ver 1.92 driver for a Intel64 cpu I would appreciate it.
 
Hi guys, seems i am having the same issue with my realtek's and my pfsense box, I am a newb when it comes to freeBSD and was wodnering if anyoen has the pre-compiled fixed drivers listed above int his thread for either 10.3 or 11 ? I still have to figure out how to use the pre-compiled patches but i think that might be easier than tryiing to follow shadowcasters instructions above :) at least for me :)

thanks
sde said:
Hey Incanter, do you happen to have that kernel module and would you mind uploading to google drive or elsewhere? I say Kenny's version above but I'm guessing that is v1.92 which he was having issues with throughput. I'd like to try the 1.88 version but the Realtek site no longer has it from what I can see.
Click to expand...

thr3e said:
Hi, I came across this post while researching for the same issue. My Realtek NIC (RTL8111E) does not play well with FreeNAS 9.10-stable (which built on FreeBSD 10.3), keep auto negotiating at 100mb instead of Gigabit. I see that Realtek has released a new driver ver 1.92 on 10/25/2016, package includes the "if_re.c" and "if_rereg.h" files. From what I read I need to compile the ".c" file into a ".ko" but I'm new to Linux and I'm still a little confused about compiling.

If anyone have a copy of the compiled ".ko" with the newest Realtek ver 1.92 driver for a Intel64 cpu I would appreciate it.
Click to expand...
 
So i guess what you are saying is that my odds of finding a precompiled driver are slim eh? :)

Any idea on a good resource for me to learn how to do it myself? I guess i can run a vm with 10.3 or 11 to compile on? then add that compiled driver to my pfsense box?
 
The first step would be to ask the pfSense people to set up a way to produce updated drivers. This is a problem for many people.
Then consider setting up a VM to build your own.
Look at opnSense, the fork of pfSense, which might have support or a more aggressive release schedule or some method of distributing driver updates.
Replacing the motherboard with one that has Intel networking built in or at least has a single PCIE slot is another option.
 
wblock@ said:
Replacing the motherboard with one that has Intel networking built in or at least has a single PCIE slot is another option.
Click to expand...
May you also be so kind to point out what chipset out of Intel network products exactly should I look foor while buying embedded motherboard? I am having bad luck finding mini-pcie on intel site, and it's not always clear how to identify "proper" motherboards. More to say, whatever intel chips are covered by "em" driver are not so good, next some will say that probably I need to look for "igb" one, and in the end of the day I will end up with a lot of advices and OS without network connectivity at all.
 
Not mini-PCIE, but full-sized, normal PCIE. The em(4) and igb(4) drivers support everything I've tried, which admittedly isn't a huge variety. Recently, that has been i217V, I219, and I210.

Come to think of it, mini-PCIE boards should work, it's just a different form factor. But a software build-your-own-driver setup is less expensive and more likely to give a good result.

Still, see what opnSense has. It sounded like they forked off of pfSense because of things moving too slow, and this might be the very sort of thing they would cover.

You know, looking at that motherboard again, it has an X1 PCIE slot on it.
 
I'm using latest 11-stable, it won't probably help you. Setting sysctl net.inet.tcp.tso=0 may help mitigate your problem, at least I was able to get 2 month of uninterruptible usage somehow with build-in driver.
 
I'm also on 11 stable, running pfsense beta 2.4 since i tried to see if ti would help with my issue. It has not :) have to limit my NIC to 100Mpbs

I will try and compile my own.. somehow :)


Shadowcaster said:
I'm using latest 11-stable, it won't probably help you. Setting sysctl net.inet.tcp.tso=0 may help mitigate your problem, at least I was able to get 2 month of uninterruptible usage somehow with build-in driver.
Click to expand...
 
Shadowcaster said:
I'm using latest 11-stable, it won't probably help you. Setting sysctl net.inet.tcp.tso=0 may help mitigate your problem, at least I was able to get 2 month of uninterruptible usage somehow with build-in driver.
Click to expand...

So i compiled my own driver in 10.3, copied it over to my pfsense box , used kldload to load it. seems to have loaded with no errors?

I cannot get the version of the driver such as the one posted above to show? 1.91 or 1.92 (the one i am using)


Code: 
FreeBSD 11.0-PRERELEASE #25: Tue Oct 11 15:26:52 EEST 2016

re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether 40:8d:5c:aa:aa:aa
        inet 10.10.10.10 netmask 0xffffff00 broadcast 10.10.10.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

re1: <Realtek PCIe GBE Family Controller> port 0xd000-0xd0ff mem 0x81300000-0x81300fff,0xa0000000-0xa0003fff irq 17 at device 0.0 on pci2
re1: Using Memory Mapping!
re1: Using 1 MSI-X message
re1: ASPM disabled
re1: version:1.91
re1: Ethernet address: 40:8d:5c:aa:aa:aa

This product is covered by one or more of the following patents:
US6,570,884, US6,115,776, and US6,327,625.
re1: Ethernet address: 40:8d:5c:aa:aa:aa

I added it to my boot/modules then added a line to boot.loader and now when i reboot and check with kldstat it shows it as loaded.. so is that all?
 
Binary (compiled) driver proved itself to be unstable and unuseable under freebsd 11. It ends up flapping "link down/link up" and stalls in "link down" after few hours under load. YMMV, but build-in driver behaves bad, but a little more stable for me.
It's may be pure coincidence, but open-source driver behaves better when I use "compiled" realtek driver and then reboot into kernel with open-source one without shutting down the box.

Also, I use sysctl to disable TSO: (goes into /etc/sysctl.conf )
net.inet.tcp.tso=0

Also, disabling msi, msi-x and turning off powerd may help to mitigate this issues, but may affect performance:
hw.re.msi_disable=1
hw.re.msix_disable=1
or globally
hw.pci.enable_msix=0
hw.pci.enable_msi=0
(this goes to /boot/loader.conf and powerd can be disabled in /etc/rc.conf by putting powerd_enable="NO" line)


for me
hw.re.msi_disable=1
hw.re.msix_disable=1
did give about 5% of network performance boost! (measured with iperf3). Stability is yet to be tested.
 
You must log in or register to reply here.
Back
Top