Hosting FreeBSD 12 VMs in VMware 7 results in unexpected network performance issues

[ERM Consulting]

Even this thread is 1 year old it is still an open issue to me:
We checked network performance on VMware7 with FreeBSD on two hosts using iperf3 with 10 Gbit/s nics.

Without TSO the transfer rates have been VERY POOR.
But even with TSO the transfer rates from VMware HOST to hosted FreeBSD-VM is not good.

Using on VMware host:

iperf3 -s <host-ip>

and on VMware client:

iperf3 -c <host-ip>

We get connections like

[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.14 GBytes 9.79 Gbits/sec 0 1.63 MBytes
[ 5] 1.00-2.00 sec 1.72 GBytes 14.8 Gbits/sec 0 2.00 MBytes

But when we change the direction (FreeBSD-VMware is iperf3-server with

iperf3 -s <Client-VM-ip>

and type on server:

iperf3 -c <Client-VM-ip>

We get much lower transfer rates:

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 107 MBytes 891 Mbits/sec
[ 5] 1.01-2.00 sec 245 MBytes 2.07 Gbits/sec

Transfer between both VMware hosts via hardware nic is perfect:

[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 1.04 GBytes 8.90 Gbits/sec 8634728 0.00 Bytes
[ 4] 1.00-2.00 sec 1.09 GBytes 9.38 Gbits/sec 0 0.00 Bytes

We testet this with several nic settings but at the end nothing solves this issue.
Does anyone have an idea how we could solve this issue?

 

[ERM Consulting]

But at the end all of this "performance tests" do not change any: The transfer rates between all of the four systems are nearly the same: copying 1 GB file from any of that systems to any other system via scp transfers always between 57MB/s to 101MB/s - if we use the 1Gbit interface or the 10 Gbit.

 

[ERM Consulting]

Using VMXNET3 and VMware Virtual Switch with 10 Gbit/s nics results in bitrates from 9.8 to 23.2 Gbit/s when iperf-server is startet on VMware host but only 0.8 to 2.0 Gbit/s bitrates when iperf-server is startet on VMware client.

We have generated a test protocol with 2 VMware 7 hosts and 2 FreeBSD 12 VMs that explain the test environment and the results (see attachment "VMware 7 and FreeBSD 12 10gbit LAN connection test.pdf").

We are able and willing to do further tests and provide the results if someone gives us hints how to solve the issue.

 

[SirDice]

Please don't post the same question multiple times in different threads. Posts merged.

 

[ERM Consulting]

Please don't post the same question multiple times in different threads. Posts merged.

OK and thank you for merging!
I thougt this is a general issue and should be handled in a separate thread.
I am wondering there is nobody who has an idea, explanation or comment to this abvious issue.