Slow network performance compared to linux (again)


About one Year ago I posted topic about network performance problems in freebsd compared to linux on hetzner.
I wasn't able to solve problem and I forgot about it. Lately things start getting worst.

I decided to make simple test:

1. Created to virtual machines on digitalocean. (don't want blame only hetzner for it). One Freebsd 12.0 on Ubuntu 18.04
2. I use the same server location to have similar results
3. Just install iptraf3 on both machines
4. Run test from my home network and from other networks

Results was like expected from my network download speed is very low from Freebsd server compared to Ubuntu.
This happens only for my home network. It is possible that my provider use some kind of QoS to limit traffic?
This don't depend on IP address, Maybe some other flags in TCP protocol?
Maybe there is some kind settings in freebsd that can mimic packets like it was from linux (to verify problem).

- for udp speed are the same (but very low)
- if i use parallel mode for iperf3 I can get similar results for freebsd and linux
- I tested lots of performance optimalization Freebsd network but none of them help (changing etc).

Any ideas?

~% iperf3 -t 5 -Rc
Connecting to host, port 5201
Reverse mode, remote host is sending
[  5] local port 54503 connected to port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   191 KBytes  1.56 Mbits/sec
[  5]   1.00-2.00   sec   280 KBytes  2.29 Mbits/sec
[  5]   2.00-3.00   sec   144 KBytes  1.18 Mbits/sec
[  5]   3.00-4.00   sec   184 KBytes  1.51 Mbits/sec
[  5]   4.00-5.00   sec   402 KBytes  3.29 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.05   sec  1.25 MBytes  2.08 Mbits/sec   51             sender
[  5]   0.00-5.00   sec  1.17 MBytes  1.97 Mbits/sec                  receiver

~% iperf3 -t 5 -Rc
Connecting to host, port 5201
Reverse mode, remote host is sending
[  5] local port 54516 connected to port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.76 MBytes  14.8 Mbits/sec
[  5]   1.00-2.00   sec  2.10 MBytes  17.6 Mbits/sec
[  5]   2.00-3.00   sec  2.40 MBytes  20.1 Mbits/sec
[  5]   3.00-4.00   sec  2.37 MBytes  19.8 Mbits/sec
[  5]   4.00-5.00   sec  2.28 MBytes  19.2 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec  12.0 MBytes  20.2 Mbits/sec   25             sender
[  5]   0.00-5.00   sec  10.9 MBytes  18.3 Mbits/sec                  receiver
I'm having exactly the same problem. I just loaded a linux image via USB stick and the iperf3 has run at full speed, but returning to FreeBSD it works 10 times slower than in Linux. I tested all recomendations in several posts on this forum, but nothing worked.

Any ideas? How can it be possible having FreeBSD a superior network stack than Linux? (in theory).

Thanks in advance.
Thank you SirDice for being so inclined to help others!

I'm running a 12.0-RELEASE kernel:
uname -a
FreeBSD mymachine 12.0-RELEASE-p4 FreeBSD 12.0-RELEASE-p4 GENERIC  amd64

Card model:
dmesg | grep re0
re0: <RealTek 8169/8169S/8169SB(L)/8110S/8110SB(L) Gigabit Ethernet> port 0xee00-0xeeff mem 0xfdefe000-0xfdefe0ff irq 16 at device 4.0 on pci2
re0: Chip rev. 0x10000000
re0: MAC rev. 0x00000000
miibus0: <MII bus> on re0
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 64:70:02:04:a6:a6
re0: netmap queues/slots: TX 1/256, RX 1/256

Ifconfig shows:
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:18:f3:df:3a:91
        media: Ethernet autoselect
        status: no carrier
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 64:70:02:04:a6:a6
        inet netmask 0xffffff00 broadcast
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tap0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:bd:28:48:f7:00
        inet6 fe80::2bd:28ff:fe48:f700%tap0 prefixlen 64 scopeid 0x4
        inet netmask 0xffffff00 broadcast
        groups: tap
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 50162

The system has one wireless card (not configured) and one vpn configured with OpenVPN.

The route command shows:
route show default
   route to: default
destination: default
       mask: default
        fib: 0
  interface: re0
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0

And netstat:
netstat -r
Routing tables

Destination        Gateway            Flags     Netif Expire
default          UGS         re0      link#4             U          tap0         link#4             UHS         lo0
localhost          link#3             UH          lo0     link#2             U           re0      link#2             UHS         lo0

Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
localhost          link#3             UH          lo0
::ffff:  localhost          UGRS        lo0
fe80::/10          localhost          UGRS        lo0
fe80::%lo0/64      link#3             U           lo0
fe80::1%lo0        link#3             UHS         lo0
fe80::%tap0/64     link#4             U          tap0
fe80::2bd:28ff:fe4 link#4             UHS         lo0
ff02::/16          localhost          UGRS        lo0

There are any other details you would like to consider?

Thanks a lot in advance.
Well, I'm upgraded to 12.1-RELEASE and the problems remain the same.

I'm using a Realtek ethernet pci card so I can only use the re0 interface as em0 interface is a wireless one, not configured yet.

The most surprising for me is that iperf3 between local machines runs at full 1000mbit/sec speed (regardless of the direction: FreeBSD to Linux or Linux to FreeBSD), but when downloading from Internet, the Linux machine can go at maximum speed while FreeBSD only at one tenth of such maximum.

Any further ideas?

Thanks in advance.