Solved PF / NAT uploads exponentially faster than downloads.

ranvel

ranvel

Hello!

I'm a PF novice and I'm trying to get a setup working.

I have a Mobile Phone <-> Digital Ocean VPN with Unbound DNS... so there are a lot of moving parts.

I know that these are famous last words, but: I think it isn't DNS.

Right now, I'm reliably getting around 150 - 200 Kbps DOWN and 70 - 100 Mbps UP.

I'd really like to know how to fix this, but my true desire is getting guidance on trying to solve an issue involving a massive bottleneck in the future. I found the PF config on another site so it's possible that others will follow my footsteps and run into the same issue I have. I tried to use tcpdump ( tcpdump -n -e -ttt -i pflog0) to analyse the log traffic, but the log quantity was too great and I wasn't capable of making sense of it.

Thanks!

Here is my /etc/pf.conf file:

Code: 
# default openvpn settings for the client network
vpn_net = "10.8.0.0/24"
ext_if = "vtnet0"
vpn_if = "tun0"
ext_ip = "123.123.123.2" #Example internet-facing IP

# allowed inbound ports (services hosted by this machine)
inbound_tcp_services = "{ ssh, domain, http, https, auth, openvpn }"
inbound_udp_services = "{ dhcpv6-client, domain, openvpn }"

# politely send TCP RST for blocked packets. The alternative is
# "set block-policy drop", which will cause clients to wait for a timeout
# before giving up.
set block-policy return

# log only on the external interface
# set loginterface $ext_if

# skip all filtering on localhost
set skip on lo

# reassemble all fragmented packets before filtering them
scrub in on $ext_if all fragment reassemble

# route traffic from VPN interface out to the internet
nat on ! $vpn_if from $vpn_net to any -> $ext_ip

# block forged client IPs (such as private addresses from WAN interface)
antispoof for $ext_if

# default behavior: block all traffic
block all

# all traffic through VPN interface is assumed to be safe
pass quick on $vpn_if

# allow all icmp traffic (like ping)
pass quick on $ext_if proto icmp all
pass quick on $ext_if proto icmp6 all

# allow incoming traffic to services hosted by this machine
pass in quick on $ext_if proto tcp to port $inbound_tcp_services
pass in quick on $ext_if proto udp to port $inbound_udp_services

# allow all outgoing traffic
pass out quick on $ext_if
 
A quick update in case anyone else comes up against this issue. I was able to resolve this by also removing `rxcsum` from the device's options.

There are still a bunch more options, but I've gone from 150 Kbps downstream to 20 Mbps downstream and now I'm getting into diminishing returns on my investment of time.

Digital Ocean has a bunch of options on the vtnet0 device and I bet that I could get 100 Mbps downstream by disabling `VLAN_HWTSO` , but I'm getting errors when I try that:

Code: 
root@vpn #> ifconfig vtnet0 -VLAN_HWTSO                                                                                                                                 ~
ifconfig: -VLAN_HWTSO: bad value
root@vpn #> ifconfig vtnet0 -vlan_hwtso                                                                                                                                 ~
ifconfig: -vlan_hwtso: bad value
root@vpn #>

Here are the options I have for `vtnet0`:

Code: 
options=6c00ba<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

At this point, I don't consider this issue fixed, but definitely solved.
 
Yeah, hardware offloading options with NICs that are emulated/implemented in software is a very bad idea. Unfortunately the defaults for the vtnet devices are still not sane it seems.
 
You must log in or register to reply here.
Back
Top