FreeBSD Friends,
I have two 12.1-RELEASE computers, named star and joeking. Both use LACP with multiple Gigabit Ethernet ports.
Symptom
Both star and joeking have the same symptom.
When I run two simultaneous tests with benchmarks/iperf (from two separate computers), the tests report ~450 Mbit/sec each.
I was hoping to see them report ~900 Mbit/sec each (full Gigabit Ethernet).
Questions
star has two Broadcoms attached to "X," an unknown Ethernet switch (I could find out if it really matters).
joeking has four Intels attached to a Cisco Catalyst 2960.
star's rc.conf:
star's lagg0
joeking's rc.conf:
joeking's lagg0
I have two 12.1-RELEASE computers, named star and joeking. Both use LACP with multiple Gigabit Ethernet ports.
Symptom
Both star and joeking have the same symptom.
When I run two simultaneous tests with benchmarks/iperf (from two separate computers), the tests report ~450 Mbit/sec each.
I was hoping to see them report ~900 Mbit/sec each (full Gigabit Ethernet).
Questions
- Is it reasonable for me to expect LACP to distribute a mere two flows across separate links?
- If a greater number of flows is required before LACP "kicks in," how can I find out how many?
- Does anybody know how to tune LACP to evenly distribute a handful of flows?
- I tried swapping the iperf client/server roles (making the clients the servers); 'no change.
- I tried using iperf's TCP mode; 'no change.
- I tried using two different server-side port numbers with iperf; 'no change.
- I tried putting
net.link.lagg.default_use_flowid=1
in /etc/sysctl.conf and rebooting; 'no change.
star has two Broadcoms attached to "X," an unknown Ethernet switch (I could find out if it really matters).
joeking has four Intels attached to a Cisco Catalyst 2960.
star's rc.conf:
Code:
ifconfig_bge0="up"
ifconfig_bge1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport bge0 laggport bge1"
vlans_lagg0="11 20 21 37 57 58 59 239"
ifconfig_lagg0_11="up"
ifconfig_lagg0_20="up"
ifconfig_lagg0_21="up"
ifconfig_lagg0_37="inet 172.16.37.194 netmask 255.255.255.0"
ifconfig_lagg0_57="inet 172.16.57.101 netmask 255.255.255.0"
ifconfig_lagg0_58="up"
ifconfig_lagg0_59="up"
ifconfig_lagg0_239="inet 172.16.239.72 netmask 255.255.255.0"
star's lagg0
Code:
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 40:8d:5c:67:73:04
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=10<LACP_STRICT>
flowid_shift: 16
lagg statistics:
active ports: 2
flapping: 0
lag id: [(8000,40-8D-5C-67-73-04,00CB,0000,0000),
(8000,58-97-1E-F7-14-80,0001,0000,0000)]
laggport: bge0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,40-8D-5C-67-73-04,00CB,8000,0001),
(8000,58-97-1E-F7-14-80,0001,8000,012F)]
laggport: bge1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,40-8D-5C-67-73-04,00CB,8000,0002),
(8000,58-97-1E-F7-14-80,0001,8000,0130)]
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
joeking's rc.conf:
Code:
ifconfig_igb0="up"
ifconfig_igb1="up"
ifconfig_igb2="up"
ifconfig_igb3="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 laggport igb2 laggport igb3"
vlans_lagg0="11 20 21 37 57 58 59 239"
ifconfig_lagg0_11="up"
ifconfig_lagg0_20="up"
ifconfig_lagg0_21="up"
ifconfig_lagg0_37="up"
ifconfig_lagg0_57="inet 172.16.57.239 netmask 255.255.255.0"
ifconfig_lagg0_58="up"
ifconfig_lagg0_59="up"
ifconfig_lagg0_239="up"
joeking's lagg0
Code:
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 90:e2:ba:39:81:e0
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=10<LACP_STRICT>
flowid_shift: 16
lagg statistics:
active ports: 4
flapping: 0
lag id: [(8000,90-E2-BA-39-81-E0,018B,0000,0000),
(8000,F4-7F-35-BF-78-80,0001,0000,0000)]
laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,90-E2-BA-39-81-E0,018B,8000,0001),
(8000,F4-7F-35-BF-78-80,0001,8000,0124)]
laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,90-E2-BA-39-81-E0,018B,8000,0002),
(8000,F4-7F-35-BF-78-80,0001,8000,0122)]
laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,90-E2-BA-39-81-E0,018B,8000,0003),
(8000,F4-7F-35-BF-78-80,0001,8000,0123)]
laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,90-E2-BA-39-81-E0,018B,8000,0004),
(8000,F4-7F-35-BF-78-80,0001,8000,0125)]
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>