Solved Extremely slow performance, but only in one direction

zilti · Jul 17, 2025

I have the following setup:

Internet <--> Gateway <--wireguard--> Router <--> Server

I noticed that I get extremely bad performance for requests from the Internet to the Server, but the other way around, the performance is great.

Weirdly enough, I also get good performance from Gateway to Server, and from Internet to Gateway. So, something must be amiss on the translation in between, but I have no idea what. Does anyone have a clue what I could try? I did play around with MTU and MSS values, but that made no difference.

Here are the iperf3 results, listed as <client> to <server>:

Internet to Server:

Code:

Connecting to host REDACTED, port 5201
[  5] local 10.0.0.10 port 34132 connected to REDACTED port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  0.00 Bytes  0.00 bits/sec   28   5.27 KBytes      
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec   10   3.95 KBytes      
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    4   5.27 KBytes      
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec   11   2.63 KBytes      
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec    9   2.63 KBytes      
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec   10   2.63 KBytes      
[  5]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec    6   3.95 KBytes      
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    8   2.63 KBytes      
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    5   1.32 KBytes      
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec    9   2.63 KBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   512 KBytes   419 Kbits/sec  100            sender
[  5]   0.00-10.04  sec   384 KBytes   313 Kbits/sec                  receiver

Server to Internet:

Code:

Connecting to host ping.online.net, port 5202
[  5] local 10.0.0.50 port 63061 connected to 51.158.1.21 port 5202
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.01   sec  11.1 MBytes  92.7 Mbits/sec  430    576 KBytes      
[  5]   1.01-2.01   sec  12.6 MBytes   106 Mbits/sec    1    395 KBytes      
[  5]   2.01-3.01   sec  13.0 MBytes   109 Mbits/sec    0    440 KBytes      
[  5]   3.01-4.03   sec  14.5 MBytes   119 Mbits/sec    0    470 KBytes      
[  5]   4.03-5.02   sec  14.8 MBytes   125 Mbits/sec    0    488 KBytes      
[  5]   5.02-6.00   sec  15.0 MBytes   128 Mbits/sec    0    496 KBytes      
[  5]   6.00-7.01   sec  15.6 MBytes   129 Mbits/sec    0    498 KBytes      
[  5]   7.01-8.01   sec  15.2 MBytes   129 Mbits/sec    0    500 KBytes      
[  5]   8.01-9.00   sec  15.2 MBytes   129 Mbits/sec    0    502 KBytes      
[  5]   9.00-10.02  sec  14.6 MBytes   121 Mbits/sec   26    372 KBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.02  sec   142 MBytes   119 Mbits/sec  457            sender
[  5]   0.00-10.08  sec   141 MBytes   117 Mbits/sec                  receiver

Gateway to Server:

Code:

Connecting to host 10.0.0.50, port 5201
[  5] local 172.16.0.1 port 59950 connected to 10.0.0.50 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.03   sec  2.62 MBytes  21.4 Mbits/sec    0    107 KBytes      
[  5]   1.03-2.03   sec  7.00 MBytes  58.7 Mbits/sec    0    215 KBytes      
[  5]   2.03-3.01   sec  11.5 MBytes  97.9 Mbits/sec    0    323 KBytes      
[  5]   3.01-4.01   sec  15.9 MBytes   134 Mbits/sec    0    431 KBytes      
[  5]   4.01-5.01   sec  20.0 MBytes   168 Mbits/sec    0    539 KBytes      
[  5]   5.01-6.01   sec  15.6 MBytes   131 Mbits/sec   18    314 KBytes      
[  5]   6.01-7.01   sec  13.8 MBytes   115 Mbits/sec    0    353 KBytes      
[  5]   7.01-8.01   sec  14.8 MBytes   124 Mbits/sec    0    378 KBytes      
[  5]   8.01-9.01   sec  16.2 MBytes   136 Mbits/sec    0    391 KBytes      
[  5]   9.01-10.00  sec  16.5 MBytes   139 Mbits/sec    0    412 KBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   134 MBytes   112 Mbits/sec   18            sender
[  5]   0.00-10.03  sec   133 MBytes   111 Mbits/sec                  receiver

Server to Gateway:

Code:

Connecting to host 172.16.0.1, port 5201
[  5] local 10.0.0.50 port 18143 connected to 172.16.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.01   sec  12.9 MBytes   107 Mbits/sec  269    358 KBytes      
[  5]   1.01-2.01   sec  15.0 MBytes   125 Mbits/sec    0    407 KBytes      
[  5]   2.01-3.00   sec  13.9 MBytes   118 Mbits/sec    1    271 KBytes      
[  5]   3.00-4.01   sec  13.1 MBytes   109 Mbits/sec    0    312 KBytes      
[  5]   4.01-5.01   sec  14.0 MBytes   117 Mbits/sec    0    339 KBytes      
[  5]   5.01-6.01   sec  14.9 MBytes   125 Mbits/sec    0    355 KBytes      
[  5]   6.01-7.01   sec  14.9 MBytes   125 Mbits/sec    0    374 KBytes      
[  5]   7.01-8.00   sec  14.8 MBytes   125 Mbits/sec    0    401 KBytes      
[  5]   8.00-9.00   sec  10.8 MBytes  90.1 Mbits/sec   32    192 KBytes      
[  5]   9.00-10.00  sec  9.12 MBytes  76.5 Mbits/sec    0    218 KBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   133 MBytes   112 Mbits/sec  302            sender
[  5]   0.00-10.02  sec   132 MBytes   111 Mbits/sec                  receiver

Internet to Gateway:

Code:

Connecting to host REDACTED, port 5201
[  5] local 10.0.0.10 port 35690 connected to REDACTED port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  15.8 MBytes   132 Mbits/sec   59    421 KBytes      
[  5]   1.00-2.00   sec  15.4 MBytes   129 Mbits/sec    1    317 KBytes      
[  5]   2.00-3.00   sec  14.5 MBytes   122 Mbits/sec    0    349 KBytes      
[  5]   3.00-4.00   sec  14.5 MBytes   122 Mbits/sec    1    328 KBytes      
[  5]   4.00-5.00   sec  14.5 MBytes   122 Mbits/sec    1    342 KBytes      
[  5]   5.00-6.00   sec  15.4 MBytes   129 Mbits/sec    0    449 KBytes      
[  5]   6.00-7.00   sec  14.9 MBytes   125 Mbits/sec    2    449 KBytes      
[  5]   7.00-8.00   sec  14.8 MBytes   124 Mbits/sec    1    361 KBytes      
[  5]   8.00-9.00   sec  15.4 MBytes   129 Mbits/sec    0    402 KBytes      
[  5]   9.00-10.00  sec  13.8 MBytes   115 Mbits/sec   25    232 KBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   149 MBytes   125 Mbits/sec   90            sender
[  5]   0.00-10.02  sec   146 MBytes   123 Mbits/sec                  receiver

mumu · Jul 18, 2025

I have this for Wireguard's sake on a VPS.. added this to /boot/loader.conf and reboot..

Code:

hw.vtnet.lro_disable="1"
hw.vtnet.tso_disable="1"
hw.vtnet.csum_disable="1"

Replace 'vtnet' with your device name..or ifconfig if_name -tso -lro and see if that helps..

zilti · Wednesday at 12:03 AM

Thank you, that improved the speed to about 7-8 Mbits - it did not fix things, but it's a start...

zilti · Wednesday at 12:08 AM

Alright, so, I get a further significant improvement by adding scrub fragment reassemble max-mss 1360 no-df random-id to the Gateway's pf.conf. Neat! This might be the solution, but I'll watch this for a while and do some further testing.

Solved Extremely slow performance, but only in one direction

zilti

mumu

zilti

zilti