PF WireGuard and PF forwarding

[JC Denton]

I'd like to replicate a set of Linux systems I have with FreeBSD. Currently, there is an external "VPS," a local "router," and a local set of service hosts. The VPS systems use their public IPs and forward traffic bound for service ports (e.g., 80, 443, etc.) to the local router, where the traffic is then sent onto a set of hosts that share the same subnet as the router. There's a WireGuard connection between the VPS and the local router. The local router has a WireGuard allow for 0.0.0.0/0, but no automatic routing table changes (to avoid breaking the default route). The local service hosts send traffic back to the local router (they themselves don't use WireGuard). Try as I might, I can't replicate this with PF and am wondering if someone can easily translate my DNATs and MASQUERADES.

This particularly example has inbound TCP/8000 and outbound TCP/25 with both IPv4 and IPv6. Would really love to be able to do this same setup, but with FreeBSD

VPS:

[Interface]
Address = 10.30.1.1/32, fc00:beef:cafe::1/32
PrivateKey = PRIVATE KEY
ListenPort = 51820
Table = off
PostUp = /etc/network/policy.sh

[Peer]
PublicKey = PUBLIC KEY
PresharedKey = PSK
AllowedIPs = 0.0.0.0/0, ::/0

/etc/network/policy.sh:
#!/bin/sh

# wireguard tunnel routing
ip -4 route add 10.30.1.2/32 dev wg0
ip -6 route add fc00:beef:cafe::2/128 dev wg0

# NAT rules
iptables -A PREROUTING -t nat -i enp1s0 -d VPS IP -p tcp --dport 8000 -j DNAT --to-destination 10.30.1.2:8000
ip6tables -A PREROUTING -t nat -i enp1s0 -d VPS IP6 -p tcp --dport 8000 -j DNAT --to-destination '[fc00:beef:cafe::2]:8000'

iptables -A POSTROUTING -t nat -s 10.30.1.2 -p tcp --dport 25 -j MASQUERADE
ip6tables -A POSTROUTING -t nat -s fc00:beef:cafe::2 -p tcp --dport 25 -j MASQUERADE

Local router:

[Interface]
Address = 10.30.1.2/32, fc00:beef:cafe::2/128
PrivateKey = PRIVATE KEY
Table = off
PostUp = /etc/network/policy.sh

[Peer]
PublicKey = PUBLIC KEY
PresharedKey = PSK
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = VPS IP:51820
PersistentKeepalive = 25

/etc/network/policy.sh:
#!/bin/sh

# wireguard tunnel routing
ip -4 route add 10.30.1.1/32 dev wg0
ip -6 route add fc00:beef:cafe::1/128 dev wg0

# wireguard policy routing
ip -4 route add default dev wg0 table wireguard
ip -4 rule add pref 1000 fwmark 4 lookup wireguard

ip -6 route add default dev wg0 table wireguard6
ip -6 rule add pref 1000 fwmark 6 lookup wireguard6

# policy marking
iptables -t mangle -A PREROUTING -s 10.30.0.2 \! -d 10.30.0.254 -p tcp --sport 8000 -j MARK --set-mark 4
ip6tables -t mangle -A PREROUTING -s SERVICES V6 \! -d ROUTER V6 -p tcp --sport 8000 -j MARK --set-mark 6

iptables -t mangle -A PREROUTING -s 10.30.0.2 \! -d 10.30.0.254 -p tcp --dport 25 -j MARK --set-mark 4
ip6tables -t mangle -A PREROUTING -s SERVICES V6 ! -d ROUTER V6 -p tcp --dport 25 -j MARK --set-mark 6

# forwarding
iptables -A PREROUTING -t nat -i wg0 -p tcp --dport 8000 -j DNAT --to-destination 10.30.0.2:8000
ip6tables -A PREROUTING -t nat -i wg0 -p tcp --dport 8000 -j DNAT --to-destination '[SERVICES V6]:8000'

# destination masquerading
iptables -A POSTROUTING -t nat -s 10.30.0.2 -p tcp --dport 25 -j MASQUERADE
ip6tables -A POSTROUTING -t nat -s SERVICES V6 -p tcp --dport 25 -j MASQUERADE

Local services:
Uses local router as a gateway...

 

[kjeacle]

I do some routing with pf over Wireguard tunnels but I'm not familiar enough with iptables to know what those commands are doing to translate. For example, I route traffic from a specific host (192.168.1.49) on my LAN out through a Wireguard tunnel (wg1) on a FreeBSD host. This is the pf config on that gateway host:

scrub on wg1 max-mss 1380
nat on wg1 from 192.168.1.49 to any -> (wg1)
pass in quick on re0 route-to (wg1 10.0.0.1) from 192.168.1.49 to any

Perhaps describe what your iptables commands are doing, or what particular aspect you can't get working, and someone more familiar with pf can describe how that would be done.

 

[SirDice]

I can't replicate this with PF and am wondering if someone can easily translate my DNATs and MASQUERADES.

Why use NAT at all? You can probably solve this issue with routing alone.

 

[JC Denton]

Why use NAT at all? You can probably solve this issue with routing alone.

The "local" boxes are behind NAT and don't have public IPs of their own.

The VPS takes in connections on TCP/8000 and sends them to the local router on its WireGuard interface. That system then takes packets inbound on wg0 for TCP/8000 and sends them on the service hosts in the local subnet. For TCP/25, it's the reverse direction, so the local system can connect out to port 25 (SMTP) as the VPS's public IP.