I have the following setup:
Internet <--> Gateway <--wireguard--> Router <--> Server
Now I noticed that while the speeds are great from Server to Internet, the speeds the other way around are atrociously bad. As a next step I disabled the firewalls on Gateway and Server, and ran iperf3. Here are the results:
Server as iperf3 server, Gateway as client:
Server as iperf3 client, Gateway as server:
As you can imagine, I was quite stumped as to what I could do to improve the situation.
What solved it was to set an MTU/MSS of 1360 everywhere.
In /etc/pf.conf:
In wg0.conf in [Interface]:
And of course at the respective spots of the OPNsense router.
I hope this shows up on Google and helps someone with the same issue.
Internet <--> Gateway <--wireguard--> Router <--> Server
Now I noticed that while the speeds are great from Server to Internet, the speeds the other way around are atrociously bad. As a next step I disabled the firewalls on Gateway and Server, and ran iperf3. Here are the results:
Server as iperf3 server, Gateway as client:
Code:
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 172.16.0.1, port 56542
[ 5] local 10.0.0.50 port 5201 connected to 172.16.0.1 port 63819
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 256 KBytes 2.08 Mbits/sec
[ 5] 1.01-2.00 sec 256 KBytes 2.12 Mbits/sec
[ 5] 2.00-3.02 sec 256 KBytes 2.07 Mbits/sec
[ 5] 3.02-4.01 sec 256 KBytes 2.12 Mbits/sec
[ 5] 4.01-5.01 sec 128 KBytes 1.04 Mbits/sec
[ 5] 5.01-6.00 sec 256 KBytes 2.12 Mbits/sec
[ 5] 6.00-7.02 sec 256 KBytes 2.07 Mbits/sec
[ 5] 7.02-8.01 sec 256 KBytes 2.12 Mbits/sec
[ 5] 8.01-9.01 sec 256 KBytes 2.09 Mbits/sec
[ 5] 9.01-10.00 sec 128 KBytes 1.05 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.03 sec 2.25 MBytes 1.88 Mbits/sec receiverServer as iperf3 client, Gateway as server:
Code:
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 172.16.0.2, port 34960
[ 5] local 172.16.0.1 port 5201 connected to 172.16.0.2 port 52896
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 5.25 MBytes 44.0 Mbits/sec
[ 5] 1.00-2.03 sec 6.50 MBytes 53.0 Mbits/sec
[ 5] 2.03-3.01 sec 9.88 MBytes 84.5 Mbits/sec
[ 5] 3.01-4.03 sec 14.2 MBytes 117 Mbits/sec
[ 5] 4.03-5.01 sec 12.9 MBytes 110 Mbits/sec
[ 5] 5.01-6.01 sec 11.0 MBytes 92.3 Mbits/sec
[ 5] 6.01-7.01 sec 12.2 MBytes 103 Mbits/sec
[ 5] 7.01-8.01 sec 13.1 MBytes 110 Mbits/sec
[ 5] 8.01-9.01 sec 14.4 MBytes 121 Mbits/sec
[ 5] 9.01-10.01 sec 14.9 MBytes 125 Mbits/sec
[ 5] 10.01-10.05 sec 512 KBytes 111 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.05 sec 115 MBytes 95.9 Mbits/sec receiverAs you can imagine, I was quite stumped as to what I could do to improve the situation.
What solved it was to set an MTU/MSS of 1360 everywhere.
In /etc/pf.conf:
scrub fragment reassemble max-mss 1360 no-df random-idIn wg0.conf in [Interface]:
MTU = 1360And of course at the respective spots of the OPNsense router.
I hope this shows up on Google and helps someone with the same issue.