Solved FreeBSD 10, unable to reach 1 Gbps

korund

korund

Hi all,

I have installed FreeBSD 10-RELEASE and noticed that I'm unable to reach 1 Gbps. To measure bandwidth I use iperf version 2.0.5 on both server and host. Results vary every time, but in average we have something like:

Code: 
[  4] local 127.0.0.1 port 46304 connected with 192.168.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   778 MBytes   602 Mbits/sec
[  4] local 192.168.1.12 port 5001 connected with 192.168.1.100 port 15292
[  4]  0.0-10.0 sec  1.08 GBytes   923 Mbits/sec
I consider result from host to server (923 Mbits/sec) as acceptable while from server to host (602 Mbits/sec) is way below my expectations.
Also, I have the same unimpressive speed transferring files via CIFS to Windows host.

Hardware on server side:
Code: 
Intel Pentium G3420 @ 3.2 Ghz
Asus p9d-x mb
8GB ECC 1600 Mhz RAM
Inbuilt dual Intel I210AT interfaces
Hardware on client side:
Code: 
Very-powerful-workstation 
Intel Gigabit CT Desktop Adapter (PCI-E)

Any ideas how to improve speed between these two?
 
Jumbo frames usually improve speed somewhat. These can be set in /etc/rc.conf:
Code: 
ifconfig_em0="SYNCDHCP mtu 6122"

Catches:
  1. The MTU size is also set in the route, so either manually change it with route(8), or reboot.
  2. The switch must also support jumbo frames. Most do. Certainly a gigabit switch should.
 
wblock@ said:
These can be set in /etc/rc.conf:
Click to expand...

Adjusted rc.conf and now it looks like this:

Code: 
ifconfig_igb0="inet 192.168.1.100 netmask 255.255.255.0 SYNCDHCP mtu 6122"
Rebooting... And even worse:

Code: 
[  4] local 127.0.0.1 port 50324 connected with 192.168.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   490 MBytes   411 Mbits/sec
[  4] local 192.168.1.12 port 5001 connected with 192.168.1.100 port 44424
[  4]  0.0-10.0 sec   718 MBytes   602 Mbits/sec
 
Sorry, I should have mentioned that MTU needs to be increased on both ends. But there may be other variables also.
 
It depends on the switch, too. My cheap switch only handles jumbo frames up to 6122 bytes. Intel cards can go much higher, but it doesn't help if the switch can't support them.
 
There are a some interesting articles about FreeBSD performance tuning on calomel.org.

I'd also recommend to perform measurements under real working conditions and to avoid artificial test (like using ${WHATEVER}perf or piping /dev/zero to /dev/null via nc()).
 
wblock@ said:
It depends on the switch, too. My cheap switch only handles jumbo frames up to 6122 bytes. Intel cards can go much higher, but it doesn't help if the switch can't support them.
Click to expand...

Thanks, I'll try to connect two hosts via crossover cable directly. According to datasheet this switch support up to 10 Kb jumbo frame. TP-Link TL-SG1008.

worldi said:
There are a some interesting articles about FreeBSD performance tuning on calomel.org.

I'd also recommend to perform measurements under real working conditions and to avoid artificial test (like using ${WHATEVER}perf or piping /dev/zero to /dev/null via nc()).
Click to expand...

As stated in 1st post iperf was used to measure bandwidth. Will check calomel.org.
 
Also keep in mind that FreeBSD 10.0 still has quite a bit of debugging options turned on. Those can effect performance too.
 
wblock@ said:
It depends on the switch, too. My cheap switch only handles jumbo frames up to 6122 bytes. Intel cards can go much higher, but it doesn't help if the switch can't support them.
Click to expand...
It may be useful to run the test to the loopback address on the same machine. This will give you an idea of the theoretical maximum performance, without any hardware limitations. If you get low speeds on the loopback interface, there's no point in trying larger packet sizes or fancy hardware on the real interface.

I get 26 gigabits/sec using the loopback interface:
Code: 
(1:139) gate:/tmp# iperf -c 127.0.0.1
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 42.0 KByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 19285 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  30.3 GBytes  26.0 Gbits/sec

SirDice said:
Also keep in mind that FreeBSD 10.0 still has quite a bit of debugging options turned on. Those can effect performance too.
Click to expand...
That should have all been turned off 3.5 months ago, in r256283
 
korund said:
Adjusted rc.conf and now it looks like this:

Code: 
ifconfig_igb0="inet 192.168.1.100 netmask 255.255.255.0 SYNCDHCP mtu 6122"
Click to expand...
So, what exactly are you trying to do? You're setting a static IP address and then you do DHCP? Use one or the other, not both.
 
Terry_Kennedy said:
It may be useful to run the test to the loopback address on the same machine. This will give you an idea of the theoretical maximum performance, without any hardware limitations. If you get low speeds on the loopback interface, there's no point in trying larger packet sizes or fancy hardware on the real interface.

I get 26 gigabits/sec using the loopback interface:
Code: 
(1:139) gate:/tmp# iperf -c 127.0.0.1
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 42.0 KByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 19285 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  30.3 GBytes  26.0 Gbits/sec
Click to expand...

Just checked and got 32 Gbits/sec on the loopback.

Code: 
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 55.8 KByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 34824 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  37.3 GBytes  32.1 Gbits/sec

SirDice said:
So, what exactly are you trying to do? You're setting a static IP address and then you do DHCP? Use one or the other, not both.
Click to expand...

Sorry for confusion, I've just pasted part of config from above. This network use static IP's.

Code: 
ifconfig_igb0="inet 192.168.1.100 netmask 255.255.255.0"
 
Ok, that's solved, switch is involved someway. Connected two computers via crossover cable and able to get stable >900 Mbit.

Maybe worth to avoid TP-Link's "almost gigabit" switches. Writing to server is still tiny questionable via Samba4, but I'll create topic in appropriate area for this. Thanks everybody for valuable suggestions! :i
 
You must log in or register to reply here.
Back
Top