Aggregating 2 Gb interfaces to increase bandwidth

nazari

nazari

Hi everyone,

I created a LAGG interface and use LACP laggproto on two FreeBSD routers with these commands:

Code: 
ee  /boot/loader.conf
#lagg
if_lagg_load="YES"
Code: 
R1# ifconfig lagg0 create
R1# ifconfig lagg0 up laggproto lacp laggport gbeth1 laggport gbeth2 192.168.4.114/24

R2# ifconfig lagg0 create
R2# ifconfig lagg0 up laggproto lacp laggport gbeth1 laggport gbeth2 192.168.4.115/24

Then I generated traffic with IPERF between routers, but can not send more than 900Mb traffic. Does my command have problems or can IPERF not generate 2Gb traffic? I use loadbalance laggproto but it does not work!

Thank you for all of your comments and help.
 
Combining many network interfaces with LACP does not give you a combined throughput. The benefit is when one interface is under load another can be used for subsequent transfers. See this serverfault thread for details.
 
Well, actually bonding can give something like "combined bandwidth"; at least 802.3ad says so and LACP supports it.

On the other hand there is a strong argument against it in that it needs to connect both interfaces to the same switch which obviously creates a SPOF (single point of failure) in resilience oriented settings.

One should keep in mind that the network link is only 1 part in a more complex scenario. The data transmitted must also be read from and written to somewhere, often a disk. Moreover they must be processed and diverse issues like TCP-offloading enter the equation.

Are you sure that all other factors other than the theoretical throughput (disks, ethernet cards, etc on both sides) provide the necessary bandwidth?
 
rmoe said:
Well, actually bonding can give something like "combined bandwidth"; at least 802.3ad says so and LACP supports it.
Click to expand...
For multiple sessions, yes. For single sessions, not really. The issue is that the members of the LACP group can have different propagation delays, which can lead to out-of-order packets and impact services running over UDP, for example. The various methods for dividing up traffic between the links work to ensure that a single traffic flow will not be split across multiple links but will use a single link. For example, one of my switches offers these methods:

Code: 
switch2(config-if-Po2)#hashing-mode ?
<1-7> Enter LAG hashing mode.
1 - Src MAC, VLAN, EtherType, Src module and portId.
2 - Dest MAC, VLAN, EtherType, Src module and portId.
3 - Src IP and Src TCP/UDP port.
4 - Dest IP and dest TCP/UDP port.
5 - Src/Dest MAC, VLAN, EtherType, Src MODID/port.
6 - Src/Dest IP and Src/Dest TCP/UDP port.
7 - Enhanced hashing mode.
 
Thanks,

But how to achieve this goal? The router's CPU is "Intel(R) Xeon(R) CPU E5504 @ 2.00GHz (2000.12-MHz K8-class CPU)" and many Gb interfaces. It also does not show BUS speed or BUS width with dmesg.boot and sysinfo and dmidecode commands in FreeBSD router.

Is there any other way except LACP to combine interfaces and increase bandwidth?
 
Imagine it like a train station. If you have just one large train it doesn't matter how many tracks your station has. If, however, you have multiple trains then multiple tracks increase throughput.

In lagg terms a train is a "communication", i.e. a connection that is characterized by relevant parameters such as source and target MAC and IP (incl. port AFAIK) and VLAN (if applicable). Keep in mind that lagg lives at the MAC level, so MAC (and hence lagg) knows about a "communication" because it can see the frames info.

In practical terms, e.g. transferring one large file is one train; so, lagg or not is quite the same. Where aggregation comes into play is when there are multiple "communications", i.e. when there are (for instance) multiple files transferred (without socket reuse). Then lagg can send those different "communications" through different pipes; then there are two trains using two tracks in the station.

One typical issue with lagg is applications not knowing about it and establishing a connection and keeping it open to transmit data to and fro (which seems reasonable from the application's perspective). Depending on some details this may lead to a situation where lagg thinks it's just 1 "communication". Why? Because of the way lagg is implemented. In FreeBSD a "communication" is identified by a hash over the protocol header. (And BTW lacp and loadbalance come down to roughly the same, performance-wise (but loadbalance is static)).

So, next to having machines that are powerful enough to read and write the data you'd also need to make sure your test (or actual application) actually creates many connections rather than reusing one connection over and over again.

So, in pseudocode it would be something like:
Code: 
do 100 times:
   skt = open_socket("other_side_IP", somePort)
   transmit_1MB_data(skt, data)
   close_socket(skt)
 
You must log in or register to reply here.
Back
Top