PF NAT to route Wireguard clients' traffic

aragats

aragats

I'm missing something obvious.
Why NAT doesn't work in a Wireguard server to route clients' traffic?
Code: 
ext_if="vtnet0"
wg_clients="10.40.0.0/24"

nat on $ext_if inet from $wg_clients to any -> ($ext_if)

The server's Wireguard IP address is 10.40.0.1, the client's - 10.40.0.30.
Running tcpdump() on the server and pinging 1.1.1.1 from a client:
Code: 
# tcpdump -n -i wg0 icmp
listening on wg0, link-type NULL (BSD loopback), capture size 262144 bytes
18:10:39.237432 IP 10.40.0.30 > 10.40.0.30: ICMP host 1.1.1.1 unreachable, length 92
Why the source IP in the response is the same client's IP?
When pinging the server's IP everything looks normal:
Code: 
18:33:44.209793 IP 10.40.0.30 > 10.40.0.1: ICMP echo request, id 45088, seq 3, length 64
18:33:44.209819 IP 10.40.0.1 > 10.40.0.30: ICMP echo reply, id 45088, seq 3, length 64
 
FWIW here is a full working pf.conf from a Raspberry Pi running FreeBSD and acting as a Wireguard server:

Code: 
set skip on lo0
scrub in
nat on ue0 from 10.0.0.0/24 to any -> 192.168.1.1
pass all

wg0 on server is 10.0.0.1, clients are in 10.0.0.0/24. Pi's ethernet is ue0 with IP of 192.168.1.1.
 
Did you add gateway_enable="YES" to rc.conf? Nothing will get routed if this is off.
 
SirDice said:
Did you add gateway_enable="YES" to rc.conf? Nothing will get routed if this is off.
Click to expand...
Of course, that was the first thing.
I have other FreeBSD boxes with NAT: routing jails, VMs etc. I thought, maybe there is something special with Wireguard interface...
 
This may give some clue. When pinging 1.1.1.1 the response comes from the same IP, not from the gateway 10.40.0.1:
Code: 
% ping 1.1.1.1
ping: sendto: Capabilities insufficient
92 bytes from 10.40.0.30: Destination Host Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 5c28   0 0000  40  01 123a 10.40.0.30  1.1.1.1
Also, "Capabilities insufficient" is misleading, I remember seeing a bug report regarding this message.
 
I've setup the same in another box:
Code: 
% ifconfig wg0
wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
    options=80000<LINKSTATE>
    inet 10.40.10.1 netmask 0xffffff00
    groups: wg
    nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>
pf.conf:
Code: 
ext_if="vtnet0"
wg_if="wg0"
wg_clients="10.40.10.0/24"

set skip on lo0
scrub in all

nat log on $ext_if from $wg_clients to any -> ($ext_if)

block drop log on $wg_if all
pass in log on $wg_if all
pass out log on $wg_if all
Pinging 1.1.1.1 from a client 10.40.10.26:
Code: 
# tcpdump -n -e -ttt -i pflog0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
 00:00:00.000000 rule 1/0(match): pass in on wg0: 10.40.10.26 > 10.40.10.26: ICMP host 1.1.1.1 unreachable, length 92
 00:00:00.000022 rule 2/0(match): pass out on wg0: 10.40.10.26 > 10.40.10.26: ICMP host 1.1.1.1 unreachable, length 92
The server's Wireguard IP 10.40.10.1 is not in the log at all!
 
Try to reload the pf.conf after vtnet0 is up and have address or change the NAT to point to the ext_if ip address instead of ($ext_if).
What is the ip address of vtnet0?
pfctl -sn
 
Nothing has changed after I replaced the interface with the actual IP address.
Still get "ICMP host 1.1.1.1 unreachable" message when pinging.
Code: 
# pfctl -sn
nat log on vtnet0 inet from 10.40.10.0/24 to any -> 5.2.79.92
 
server:
Code: 
# cat /usr/local/etc/wireguard/wg0.conf
[Interface]
Address = 10.40.10.1/24 
ListenPort = 51820
PrivateKey = KHakPUxxxxxxxxxxxxxxxxxxxxxxxxxxxxmTCZVc=

[Peer]
# thinkpad x1
AllowedIPs = 10.40.10.26/32
PublicKey = b2T8cGOId5hRLDjb3qkDsAG48PA0g+pRs3YrCl0weFc=

<other peers>
Code: 
% netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            5.2.79.1           UGS      vtnet0
5.2.79.0/24        link#1             U        vtnet0
5.2.79.92          link#1             UHS         lo0
10.40.10.0/24      link#3             U           wg0
10.40.10.1         link#3             UHS         lo0
10.40.10.26        link#3             UHS         wg0
127.0.0.1          link#2             UHS         lo0
client:
Code: 
# cat /usr/local/etc/wireguard/wg2.conf
[Interface]
PrivateKey = OFMWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxSoZOIXg=
Address = 10.40.10.26/24

[Peer]
PublicKey = hzdZyvrhh0AfBUcPH+VRXjgOdHqkppPRySGqyS/Cdnw=
AllowedIPs = 10.40.10.0/24
Endpoint = 5.2.79.92:51820

PersistentKeepalive = 30
Code: 
% setfib 1 netstat -rn
Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.40.10.1         UGS         wg2
10.40.10.0/24      link#9             U           wg2
 
VladiBG said:
you need to allow all traffic via the wg (0.0.0.0/0)
Click to expand...
Thank you, VladiBG !
I set 0.0.0.0/0 on the client side together with 10.40.10.0/24 on the server side.
Now I can ping 5.2.79.92 (server's external IP) from the client. However, in the routing table it's lo0.
Pinging an external host still doesn't work, but instead of "Unreachable" it just gets no replies. On the server side:
Code: 
% tcpdump -n -e -ttt -i wg0 icmp
listening on wg0, link-type NULL (BSD loopback), capture size 262144 bytes
 00:00:00.000000 AF IPv4 (2), length 88: 10.40.10.26 > 1.1.1.1: ICMP echo request, id 6051, seq 49, length 64
For some reason NAT doesn't work.
 
VladiBG said:
Your firewall is blocking the traffic on vtnet0
Click to expand...
I removed everything from my pf.conf, except the following:
Code: 
ext_if="vtnet0"
wg_if="wg0"
wg_clients="10.40.10.0/24"

set skip on lo0
scrub in all

nat log on $ext_if from $wg_clients to any -> 5.2.79.92
Still the same behavior: I can ping vtnet0 (5.2.79.92) from a WG client, but not external hosts.
 
Also, as I mentioned previously, leaving routing the same, but switching to OpenVPN makes everything working.
So, indeed, there is something special with Wireguard...
 
aragats said:
Also, as I mentioned previously, leaving routing the same, but switching to OpenVPN makes everything working.
So, indeed, there is something special with Wireguard...
Click to expand...
I just setup Wireguard and everything works on 13.2-RELEASE-p3. I have IP Forwarding enabled and the NAT rules for PF as well.
 
mumu said:
I just setup Wireguard and everything works on 13.2-RELEASE-p3. I have IP Forwarding enabled and the NAT rules for PF as well.
Click to expand...
Thanks for confirming. I set up the same configuration on another VPS, and it works too.
I'm not sure what's wrong with the original one.

I noticed strange traffic on it, and got the following message from the provider:
... layer2 network "noise" because your VPS uses a a larger VLAN shared with many other VPSes. There are many IPs within this VLAN, all adding L2 network noise.
Click to expand...
I'm not sure whether and how it might be related to my issue though.
 
You must log in or register to reply here.
Back
Top