Solved share ipsec vpn between GATEWAY and CLIENTS using IPFW and NAT

mave76 · Nov 22, 2022

Hi Everybody, I'm posting after reading many many threads on this forum.
I've this kind of configuration in which I would like to route my clients traffic throught a FreeBSD gateway connected to the corporate network via IPSec.

Code:

(Clients) : 192.168.1.0/24 -> (FBSD 13.1-RELEASE-p3) : 192.168.1.251 -> (corporate ipsec vpn) with internet public ip (X.X.X.X)

An IPSec connection is successfully established with strongswan between FreeBSD to corporate VPN server and then, after connection I've an ipsec0 point to point interface with inet 10.10.10.68:

Code:

ipsec0: flags=8151<UP,POINTOPOINT,RUNNING,PROMISC,MULTICAST> metric 0 mtu 1400
        tunnel inet 192.168.1.251 --> X.X.X.X
        inet 10.10.10.68 --> 10.10.10.68 netmask 0xff000000
        groups: ipsec
        reqid: 100
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

At this point on FreeBSD server everything works through IPSec, I can reach every host in my corporate network.
My goal is permit to clients to reach same resources via ipfw with nat and routing.

This is the relevant part of /etc/rc.conf

Code:

gateway_enable="YES"
firewall_enable="YES"
firewall_type="open"
firewall_nat_enable="YES"
firewall_script="/etc/ipfw.rules"
strongswan_enable="YES"

This is the /etc/ipfw.rules file:

Code:

#!/bin/sh

WAN="ipsec0"
LAN="em0"
CMD="/sbin/ipfw -q"

$CMD flush
$CMD nat 1 config if $WAN same_ports unreg_only reset

$CMD add 10 allow ip from any to any via $LAN
$CMD add 20 allow ip from any to any via em1 
$CMD add 30 allow ip from any to any via lo0

$CMD add 40 reass all from any to any in

# NAT rule for incomming packets.
$CMD add 100 nat 1 ip4 from any to any in recv $WAN
$CMD add 101 check-state

# Allow all other outgoing connections.
$CMD add 2000 skipto 10000 tcp from any to any out xmit $WAN setup keep-state
$CMD add 2010 skipto 10000 udp from any to any out xmit $WAN keep-state
$CMD add 2010 skipto 10000 icmp from any to any out xmit $WAN keep-state

# NAT rule for outgoing packets.
$CMD add 10000 nat 1 ip4 from any to any out xmit $WAN

# Allow anything else – just in case IPFW is not configured as open firewall.
$CMD add 65534 allow ip from any to any

On clients I added a route for the network 10.0.0.0/8 (corporate network) via 192.168.1.251 (FreeBSD) but I cannot reach any destination ip.
Starting a ping (but is true on any kind of connection) from a client to a corporate IP, I can see this from a tcpdump taken on FreeBSD:

Code:

14:52:35.136304 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 58574, seq 0, length 64
14:52:35.147521 IP 10.78.176.26 > 10.10.10.68: ICMP echo reply, id 58574, seq 0, length 64
14:52:36.157796 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 58574, seq 1, length 64
14:52:36.170614 IP 10.78.176.26 > 10.10.10.68: ICMP echo reply, id 58574, seq 1, length 64
14:52:37.206671 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 58574, seq 2, length 64
14:52:37.217198 IP 10.78.176.26 > 10.10.10.43: ICMP echo reply, id 58574, seq 2, length 64

The outbound NAT is made correctly, the request reaches the destination and the destination reply but unfortunately everything ends on the point to point interface of FreeBSD.
No packet coming back to originating IP (of network 192.168.1.0./24).

Obviously if I start the same request directly on FreeBSD it works correctly.

I tried the same configuration with PF, ipfw and natd and ipfw with kernel nat. I always get the same behavior.

Into /etc/sysctl.conf with

Code:

net.inet.ip.fw.one_pass=1 or net.inet.ip.fw.one_pass=0

nothing change.

At this point a thing, I believe I'm missing some piece of configuration about NAT but I'm struggling from days without find a suitable solution.

I'm also using the same configuration on Oracle Linux and Ubuntu using iptables with MASQUERADE and works without any issue.

I need your help if possible to figure out where I'm going wrong.
Many Thanks,

Regards.
Fabio.

SirDice · Nov 22, 2022

Turn on forwarding/routing by adding gateway_enable="YES" to rc.conf

mave76 · Nov 22, 2022

SirDice said:
Turn on forwarding/routing by adding gateway_enable="YES" to rc.conf

Thank You,
unfortunately is already configured:

Code:

gateway_enable="YES"
firewall_enable="YES"
firewall_type="open"
firewall_nat_enable="YES"
firewall_script="/etc/ipfw.rules"
strongswan_enable="YES"

Infact, if you see the tcpdump capture, the forwarding happens regularly. Is the return to originating ip that isn't working.

Regards,
Fabio.

VladiBG · Nov 22, 2022

Where is em1 facing? (rule 20)
show the output of ipfw show

SirDice · Nov 22, 2022

mave76 said:
Is the return to originating ip that isn't working.

How is the IPSec configured? Your 'left' and 'right' network definitions might be blocking the traffic.

mave76 · Nov 22, 2022

Hi VladiBG, thank you.

Sorry, my mistake leaving a rule for a non existent interface. It was for one of may attempts I have removed now

Code:

#!/bin/sh

WAN="ipsec0"
LAN="em0"
CMD="/sbin/ipfw -q"

$CMD flush
$CMD nat 1 config if $WAN same_ports unreg_only reset

$CMD add 10 allow ip from any to any via $LAN
$CMD add 30 allow ip from any to any via lo0

$CMD add 40 reass all from any to any in

# NAT rule for incomming packets.
$CMD add 100 nat 1 ip4 from any to any in recv $WAN
$CMD add 101 check-state

# Allow all other outgoing connections.
$CMD add 2000 skipto 10000 tcp from any to any out xmit $WAN setup keep-state
$CMD add 2010 skipto 10000 udp from any to any out xmit $WAN keep-state
$CMD add 2010 skipto 10000 icmp from any to any out xmit $WAN keep-state


# NAT rule for outgoing packets.
$CMD add 10000 nat 1 ip4 from any to any out xmit $WAN

# Allow anything else – just in case IPFW is not configured as open firewall.
$CMD add 65534 allow ip from any to any

Now the ipfw show output, still same issue:

Code:

00010 705 236802 allow ip from any to any via em0
00030   0      0 allow ip from any to any via lo0
00040   0      0 reass ip from any to any in
00100   0      0 nat 1 ip4 from any to any in recv ipsec0
00101   0      0 check-state :default
02000   0      0 skipto 10000 tcp from any to any out xmit ipsec0 setup keep-state :default
02010   0      0 skipto 10000 udp from any to any out xmit ipsec0 keep-state :default
02010   3    252 skipto 10000 icmp from any to any out xmit ipsec0 keep-state :default
10000   3    252 nat 1 ip4 from any to any out xmit ipsec0
65534   0      0 allow ip from any to any
65535 644 278975 deny ip from any to any

Best Regards,
Fabio.

mave76 · Nov 22, 2022

SirDice said:
How is the IPSec configured? Your 'left' and 'right' network definitions might be blocking the traffic.

HI
this is the relevant part of swanctl.conf. NAT-D is enabled and consider that the same configuration works without issue on linux:

Code:

myvpn {

        # IKE major version to use for connection.
        version = 1 

        # Local address(es) to use for IKE communication, comma separated.
        local_addrs = %any

        # Remote address(es) to use for IKE communication, comma separated.
        remote_addrs = VPNREMOTEENDPOINTIP 

        # Local UDP port for IKE communication.
        #local_port = 500

        # Remote UDP port for IKE communication.
        #remote_port = 500

        # Comma separated proposals to accept for IKE.
        proposals = aes-sha1-modp1024 

        # Virtual IPs to request in configuration payload / Mode Config.
        vips = 0.0.0.0

        # Use Aggressive Mode in IKEv1.
        aggressive = yes 

        # Set the Mode Config mode to use.
        pull = yes

        # Differentiated Services Field Codepoint to set on outgoing IKE packets
        # (six binary digits).
        # dscp = 000000

        # Enforce UDP encapsulation by faking NAT-D payloads.
        encap = yes 

        # Enables MOBIKE on IKEv2 connections.
        # mobike = yes

        # Interval of liveness checks (DPD).
        dpd_delay = 0s

        # Timeout for DPD checks (IKEV1 only).
        dpd_timeout = 0s

        # Use IKE UDP datagram fragmentation (yes, accept, no or force).
        fragmentation = yes

        # Use childless IKE_SA initiation (allow, force or never).
        childless = allow

        # Send certificate requests payloads (yes or no).
        # send_certreq = yes

        # Send certificate payloads (always, never or ifasked).
        # send_cert = ifasked

        # String identifying the Postquantum Preshared Key (PPK) to be used.
        # ppk_id =

        # Whether a Postquantum Preshared Key (PPK) is required for this
        # connection.
        # ppk_required = no

        # Number of retransmission sequences to perform during initial connect.
        # keyingtries = 1

        # Connection uniqueness policy (never, no, keep or replace).
        # unique = no

        # Time to schedule IKE reauthentication.
        # reauth_time = 0s

        # Time to schedule IKE rekeying.
        # rekey_time = 4h

        # Hard IKE_SA lifetime if rekey/reauth does not complete, as time.
        # over_time = 10% of rekey_time/reauth_time

        # Range of random time to subtract from rekey/reauth times.
        # rand_time = over_time

        # Comma separated list of named IP pools.
        # pools =

        # Default inbound XFRM interface ID for children.
        # if_id_in = 0

        # Default outbound XFRM interface ID for children.
        # if_id_out = 0

        # Whether this connection is a mediation connection.
        # mediation = no

        # The name of the connection to mediate this connection through.
        # mediated_by =

        # Identity under which the peer is registered at the mediation server.
        # mediation_peer =

        # Section for a local authentication round.

Thank You,
Fabio.

VladiBG · Nov 22, 2022

Which is your WAN interface?

Edit: I mean the parent interface of ipsec0 tunnel. Because if it's the same em0 interface you need to exempt the ipsec traffic from the rule 10

mave76 · Nov 22, 2022

VladiBG said:
Which is your WAN interface?

Edit: I mean the parent interface of ipsec0 tunnel. Because if it's the same em0 interface you need to exempt the ipsec traffic from the rule 10

Clients and FreeBSD server are on the same subnet 192.168.1.0/24 with default gateway 192.168.1.1 (router for internet). They go on internet independently
W.hen connected to vpn corporate endpoint through internet, the FreeBSD server acquire the ip provided via ike from the vpn server (10.10.10.68) on ipsec0 interface.

At this point for ipfw rules, I mean the wan interface is the ipsec0 interface. Traffic coming from 192.168.1.0/24 is natted correctly with 10.10.10.68 (tcpdump, you can see also de reply from the destination) but coming back the packet doesn't reach the source ip of the 192.168.1.0/24 originating the request.

Thank You,
Fabio.

VladiBG · Nov 22, 2022

The return traffic match rule 10 and doesn't reach your nat rule.

mave76 · Nov 22, 2022

VladiBG said:
The return traffic match rule 10 and doesn't reach your nat rule.

Removed the rule, restarted but always the same issue.
Client routing table

Code:

Internet:

Destination        Gateway            Flags           Netif Expire
default            192.168.1.1        UGScg             en0       
10                 192.168.1.251      UGSc              en0

ping from client

Code:

PING 10.78.176.26 (10.78.176.26): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4

tcpdump on server (em0: 192.168.1.251 - ipsec0 10.10.10.68) -- these are the clients requests going in timeout

Code:

17:20:33.628972 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 13490, seq 1, length 64
17:20:33.639115 IP 10.78.176.26 > 10.10.10.68: ICMP echo reply, id 13490, seq 1, length 64
17:20:34.630749 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 13490, seq 2, length 64
17:20:34.642017 IP 10.78.176.26 > 10.10.10.68: ICMP echo reply, id 13490, seq 2, length 64
17:20:35.631201 IP 10.10.10.68 > 10.78.176.26: ICMP echo request, id 13490, seq 3, length 64
17:20:35.642639 IP 10.78.176.26 > 10.10.10.68: ICMP echo reply, id 13490, seq 3, length 64

and actual ipfw rules....

Code:

#!/bin/sh

WAN="ipsec0"
LAN="em0"
CMD="/sbin/ipfw -q"

$CMD flush
$CMD nat 1 config if $WAN same_ports unreg_only reset

$CMD add 30 allow ip from any to any via lo0
$CMD add 40 reass all from any to any in

# NAT rule for incomming packets.
$CMD add 100 nat 1 ip4 from any to any in recv $WAN
$CMD add 101 check-state

# Allow all other outgoing connections.
$CMD add 2000 skipto 10000 tcp from any to any out xmit $WAN setup keep-state
$CMD add 2010 skipto 10000 udp from any to any out xmit $WAN keep-state
$CMD add 2010 skipto 10000 icmp from any to any out xmit $WAN keep-state

# NAT rule for outgoing packets.
$CMD add 10000 nat 1 ip4 from any to any out xmit $WAN

# Allow anything else – just in case IPFW is not configured as open firewall.
$CMD add 65534 allow ip from any to any

but the ping and every other connection to corporate ip works on the server

Code:

root@FBSD01VMX:~ # ping 10.78.176.26
PING 10.78.176.26 (10.78.176.26): 56 data bytes
64 bytes from 10.78.176.26: icmp_seq=0 ttl=49 time=15.722 ms
64 bytes from 10.78.176.26: icmp_seq=1 ttl=49 time=8.896 ms
64 bytes from 10.78.176.26: icmp_seq=2 ttl=49 time=9.785 ms
64 bytes from 10.78.176.26: icmp_seq=3 ttl=49 time=11.356 ms

Thank You,
Fabio.

VladiBG · Nov 22, 2022

Check the ipfw show and see if the rule 100 is matching

covacat · Nov 22, 2022

try to set
net.inet.ipsec.filtertunnel to 1

mave76 · Nov 22, 2022

covacat said:
try to set
net.inet.ipsec.filtertunnel to 1

Yeah, this parameter has fixed the issue ! Now I'll study to understand its real meaning.
Many Many Thanks to everyone helped me in this thread.

Best Regards,
Fabio.

covacat · Nov 22, 2022

iirc otherwise decapsulated ipsec packets are accepted by default, you can't filter them
man 4 ipsec

Solved share ipsec vpn between GATEWAY and CLIENTS using IPFW and NAT

mave76

SirDice

Administrator

mave76

VladiBG

SirDice

Administrator

mave76

mave76

VladiBG

mave76

VladiBG

mave76

VladiBG

covacat

mave76

covacat