FreeBSD Networking Benchmarks

The last two pages are in the 404 zone. I bought a pair of Chelsio I believe to be T5 ASICs from eBay that I'm testing. They come up beautifully with the cxgbe driver.

All four interfaces are up and pingable after giving them respective IP addresses in their respective subnet.

Code: 
cxgbe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

Running a CDP/LLDP daemon which I strongly encourage if you use good network plumbing to aide in troubleshooting.

Code: 
(*redacted *) >sh cdp n det
-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: Cisco WS-C3550-12T,  Capabilities: Switch IGMP
Interface: GigabitEthernet1/0/52,  Port ID (outgoing port): GigabitEthernet0/12
Holdtime : 137 sec

Version :
Cisco IOS Software, C3550 Software (C3550-IPSERVICESK9-M), Version 12.2(44)SE6, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 09-Mar-09 20:28 by gereddy

advertisement version: 2
Protocol Hello:  OUI=0x00000C, Protocol ID=0x0112; payload len=27, value=00000000FFFFFFFF010221FF000000000000000D658CFF00FF0000
VTP Management Domain: 'bsdworks'
Native VLAN: 1
Duplex: full
Management address(es):
  IP address: (*redacted*)

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/4,  Port ID (outgoing port): cxgbe3
Holdtime : 170 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/3,  Port ID (outgoing port): cxgbe2
Holdtime : 170 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/2,  Port ID (outgoing port): cxgbe1
Holdtime : 169 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/1,  Port ID (outgoing port): cxgbe0
Holdtime : 168 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):
 
Few months old, because ENOTIME.
I should take some time for a 13.0 vs 13.1.

Mellanox:


graph.IPv4.png


Chelsio:

graph.IPv4.png
 
So olivier you still advise disabling Hyperthreading for maximum performance?

16 Cores E5-2697A v4
Click to expand...
I disabled Hyperthreading because my router is internet facing and all those related bugs.
Plus I have plenty of real cores like you.

Are you using the Intel CPU microcode update port?
I would really like to see the differences there and how hyperthreading performs.
 
That's impressive. Unfortunately I only have 1GbE here in my small network and only have a pair of sexa-core CPUs via X5675s only giving me a dozen cores.

That's the nice thing about FreeBSD since the beginning. You can throw together some wicked packet engines if you select the right hardware.

Are there any benchmarks using Cisco FEC, LACP or PAgP on the various supported multi GbE NICS? Are there some NICS that are more lagg friendly and are better suited for lagg? If there's a better discussion thread on using lagg and the selection of multiport NICs for lagg, please link me in the right direction and I'll continue the discussion there.

I plan on running an etherchannel with my Chelsio quad port NICs and using that primarily for moving data. The pair of Chelsios I purchased do 1/10GbE so I have an upgrade path when used Cisco 10GbE switches start flooding the market at good prices. The onboard Intel em NICs will be stand by interfaces via carp and administrative access to the servers.
 
olivier said:
Disabling hyperthreading is no more as useful as it was, mainly since the introduction of machdep.hyperthreading_intr_allowed (Allow interrupts on HTT logical CPUs)
Click to expand...

Dear Olivier!

If I remember correctly, at 2018 You make ~70pages “Tuning FreeBSD for routing and firewalling” PDF presentation on AsiaBSDcon.

Do You have updated version of this presentation, modern CPUs, modern NICs?

And another one question: from which reason You not using Intel microcode patch?

P.S. Thank You for perfect research! Good luck!
 
For anyone who interested in network testing, there are another great open source tool TRex Realistic Traffic Generator https://trex-tgn.cisco.com/

[UPDATE] Some users note there are some disadvantages of this tool, so better to Googling on this forum or others network-specified...
 
Last edited:
Sergei_Shablovsky said:
For anyone who interested in network testing, there are another great open source tool TRex Realistic Traffic Generator https://trex-tgn.cisco.com/
Click to expand...
If anyone wants to take it on, feel free, but here is what I found.
forums.freebsd.org

Multi-protocol traffic generator (Seagull fork for FreeBSD, TRex ?)

Hi FreeBSD Gurus! Please recommend me a multi-Protocol traffic generator tool like Seagull. I more interesting in automation and profilings, like Saegull have. BTW Seagull is very powerful tool! Thank You all! P.S. I read on official page that “Seagull entirely coded in C++”, so may be...
forums.freebsd.org forums.freebsd.org
 
You must log in or register to reply here.
Back
Top