NAT64 with ipfw issues

Stardco · Apr 10, 2019

Hello,

I'm tying to build a gateway between a full ipv6 network to a ipv4 network. I choose to test at first ipfw (i will test taiga next).
My lab is based on BSDRP (1.92) systems inspired by "https://bsdrp.net/documentation/examples/nat64#r21". It is not exactly the same but the idea stay.

Unless I do not success...
I have a workstation in IPv6 connected to an IPv6 interface on a router and a workstation IPv4 connected to the ipv4 interface on the same router.
The pings work to the router from the workstations but not the 6 to 4 NAT between workstations.

When I look at the ipfw0 interface, I do not see any packet but the ipfw counter is incremented. I see the input traffic on the interface but nothing on the output.

This is my ipfw rules file :

Code:

#!/bin/sh
fwcmd="/sbin/ipfw"
kldstat -q -m ipfw_nat64 || kldload ipfw_nat64
${fwcmd} -f flush
${fwcmd} nat64lsn NAT64 create prefix4 10.0.148.64/26
${fwcmd} add allow log icmp6 from any to any icmp6types 135,136
${fwcmd} add nat64lsn NAT64 ip from 2001:db8:12::/64 to 64:ff9b::/96 in
${fwcmd} add nat64lsn NAT64 ip from any to 10.0.148.64/26 in
${fwcmd} add allow log ip from any to any

this is the rules counters :

Code:

[root@rTST64]~# ipfw -a list
00100  120  8168 allow log ipv6-icmp from any to any icmp6types 135,136
00200 1009 56504 nat64lsn NAT64 ip from 2001:db8:12::/64 to 64:ff9b::/96 in
00300    0     0 nat64lsn NAT64 ip from any to 10.0.148.64/26 in
00400  109 12216 allow log ip from any to any
65535    0     0 deny ip from any to any

and on the statistics :

Code:

[root@rTST64]~# ipfw nat64lsn NAT64 stats
nat64lsn NAT64
        0 packets translated from IPv6 to IPv4
        0 packets translated from IPv4 to IPv6
        0 IPv6 fragments created
        0 IPv4 fragments received
        0 output packets dropped due to no bufs, etc.
        0 output packets discarded due to no IPv4 route
        0 output packets discarded due to no IPv6 route
        0 packets discarded due to unsupported protocol
        0 packets discarded due to memory allocation problems
        1009 packets discarded due to some errors
        0 packets not matched with IPv4 prefix
        1 mbufs queued for post processing
        2 times the job queue was processed
        2 job requests queued
        0 job requests queue limit reached
        0 job requests failed due to memory allocation problems
        1 hosts allocated
        1 hosts requested
        0 host requests failed
        0 portgroups requested
        1 portgroups allocated
        2 portgroups deleted
        0 portgroup requests failed
        0 portgroups allocated for TCP
        0 portgroups allocated for UDP
        0 portgroups allocated for ICMP
        0 states created
        0 states deleted

I do not know why the packets are discarded and what is the error.

Somebody have an idea what I missed ? How can I check what happened in ipfw ?

Regards,

SirDice · Apr 10, 2019

What version of FreeBSD are you using?

Stardco · Apr 11, 2019

It is the 12.0-STABLE.

It was just trying with 12.0-RELEASE-p3. I have the same result

Stardco · Apr 11, 2019

Ok, for information, I resolved one part of the issue.

I saw in the RFC6052 (thanks to the tayga man page) that the ipv4 destination network MUST NOT be include in the RFC1918 range. So when I replaced it, I received my translated packet on my IPv4 workstation, I saw the ipv4 answer but... not the ipv6 answer back to the ipv6 workstation.

Code:

root@rFreeBSD:~ # ipfw nat64lsn all list states
2001:db8:12::1  10.0.148.78     ICMPv6          0       11.0.23.3
2001:db8:12::1  10.0.148.78     ICMPv6          4       11.0.23.3

Code:

root@rFreeBSD:~ # ipfw nat64lsn NAT64 stats
nat64lsn NAT64
        879 packets translated from IPv6 to IPv4
        879 packets translated from IPv4 to IPv6
        0 IPv6 fragments created
        0 IPv4 fragments received
        0 output packets dropped due to no bufs, etc.
        0 output packets discarded due to no IPv4 route
        0 output packets discarded due to no IPv6 route
        0 packets discarded due to unsupported protocol
        0 packets discarded due to memory allocation problems
        0 packets discarded due to some errors
        0 packets not matched with IPv4 prefix
        1 mbufs queued for post processing
        1 times the job queue was processed
        1 job requests queued
        0 job requests queue limit reached
        0 job requests failed due to memory allocation problems
        1 hosts allocated
        1 hosts requested
        0 host requests failed
        0 portgroups requested
        1 portgroups allocated
        0 portgroups deleted
        0 portgroup requests failed
        0 portgroups allocated for TCP
        0 portgroups allocated for UDP
        1 portgroups allocated for ICMP
        3 states created
        2 states deleted

Stardco · Apr 29, 2019

I find a way... a strange way but a way

The system works only if I use the same interface to do the NAT64. As a picture is better than a talk I made two schemes

Th ipfw rules are the same :

Code:

00100 allow ipv6-icmp from any to any ip6 icmp6types 135,136
00200 nat64lsn NAT64 ip from 2a01:db7:12::/64 to 64:ff9b::/96 in
00300 nat64lsn NAT64 ip from any to 11.0.148.64/26 in
00500 allow log ip from any to any
00501 allow log ip6 from any to any
65535 deny ip from any to any

It seems the nat64 through ipfw does not route the icmp reply (the 4 to 6) from an interface to another or I missed something in the ipfw configuration...
What do you think about this ?

olivier · Jun 4, 2019

Hi sir Stardco,
Because your lab are based on BSDRP 1.92, this mean you are using a FreeBSD 12-stable and there are some change here regarding NAT64.
cf the man page for man page of ipfw for 12-stable, section "IPv6/IPv4 NETWORK ADDRESS AND PROTOCOL TRANSLATION" mention this change:

Code:

After translation NAT64 translator by default sends packets through cor-
responding netisr queue. Thus translator host should be configured as
IPv4 and IPv6 router. Also this means, that a packet is handled by fire-
wall twice. First time an original packet is handled and consumed by
translator, and then it is handled again as translated packet. This
behavior can be changed by sysctl variable
net.inet.ip.fw.nat64_direct_output.

So the last rule "allow log ip from any to any" should be matched during the second pass, but there is a bug on 12-stable.
This mean that these old example can't work anymore but you can fix this by a simple:

 sysctl net.inet.ip.fw.nat64_direct_output=1

echo "sysctl net.inet.ip.fw.nat64_direct_output=1" >> /etc/sysctl.conf

Or you can complexity the rules by using tag.

mumu · Jan 16, 2023

Stardco said:
I find a way... a strange way but a way

The system works only if I use the same interface to do the NAT64. As a picture is better than a talk I made two schemes

View attachment 6446 View attachment 6447

Th ipfw rules are the same :

Code:

00100 allow ipv6-icmp from any to any ip6 icmp6types 135,136 00200 nat64lsn NAT64 ip from 2a01:db7:12::/64 to 64:ff9b::/96 in 00300 nat64lsn NAT64 ip from any to 11.0.148.64/26 in 00500 allow log ip from any to any 00501 allow log ip6 from any to any 65535 deny ip from any to any

It seems the nat64 through ipfw does not route the icmp reply (the 4 to 6) from an interface to another or I missed something in the ipfw configuration...
What do you think about this ?

Abit old I know, but wanted to let you know I got NAT64 + DNS64 + PF working on FreeBSD 11.2-RELEASE amd64. If you want more details, reply again here.

Cheers.

subnetspider · Jan 18, 2023

mumu said:
Abit old I know, but wanted to let you know I got NAT64 + DNS64 + PF working on FreeBSD 11.2-RELEASE amd64. If you want more details, reply again here.

Cheers.

I know it's not the corrent place for this, but you made me curious.
So how did you do it?

SirDice · Jan 18, 2023

mumu said:
working on FreeBSD 11.2-RELEASE amd64.

Great. But 11.2 has been End-of-Life since October 2019 and is not supported anymore. As is the 12.0 version the OP was using, that's been end-of-life since February 2020.

mumu · Jan 19, 2023

SirDice said:
Great. But 11.2 has been End-of-Life since October 2019 and is not supported anymore. As is the 12.0 version the OP was using, that's been end-of-life since February 2020.

Correct. Can't upgrade at the moment due to some legacy software my client is running. In fact if I'm given the green light to upgrade, I'd just backup everything and do a fresh install of the latest version of FreeBSD on it.

mumu · Jan 19, 2023

subnetspider said:
I know it's not the corrent place for this, but you made me curious.
So how did you do it?

Since I'm using 11.2-RELEASE which is already EoL, I did the following:-

1. Download the tayga source which is v0.9.2.
2. Patch the source using patch file from /usr/ports/net/tayga/files/, then compile and make install.
3. Configure /usr/local/etc/tayga.conf
4. Updated /etc/rc.conf with lines for tayga, local_unbound, and static routes for both IPv4/6.
5. Updated pf.conf to include a nat rule for the Tayga pool prefix and reload pf.
6. Manually add static routes for the /24 IPv4 pool prefix and the /96 IPv6 prefix. I'm using GUA for the IPv6 prefix Tayga uses.
7. Configure local_unbound and add in the dns64-prefix option and restarted local_unbound.

Tayga.conf:

Code:

# cat /usr/local/etc/tayga.conf
tun-device     tun0
ipv4-addr      192.168.250.1
ipv6-addr      2001:470:XXXX:6464:ffff::1
dynamic-pool   192.168.250.0/24
prefix         2001:470:XXXX:6464::/96
data-dir       /var/db/tayga

/etc/rc.conf:

Code:

#
ipv6_gateway_enable="YES"
gateway_enable="YES"
#
# TAYGA - interface & static routes.
ifconfig_tun0="inet6 -ifdisabled auto_linklocal up"
static_routes="nat64_ip4"
ipv6_static_routes="nat64_ip6 nat64_net"
route_nat64_ip4="-net 192.168.250.0/24 -iface tun0"
ipv6_route_nat64_ip6="-host 2001:470:XXXX:6464:ffff::1/128 -iface tun0"
ipv6_route_nat64_net="-net 2001:470:XXXX:6464::/96 -iface tun0"
# TAYGA - Configuration
tayga_enable="YES"
tayga_config_file="/usr/local/etc/tayga.conf"
tayga_tun_device="tun0"
tayga_ipv6_addr="2001:470:XXXX:6464:ffff::1"
#

ifconfig tun0:

Code:

# ifconfig tun0
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::215:5dff:fee0:2b36%tun0 prefixlen 64 scopeid 0x5
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: tun
        Opened by PID 63333

NAT rule:

Code:

nat64_net = "192.168.250.0/24"
nat on $ext_if from $nat64_net to any -> ($ext_if)

Static routes:

Code:

# netstat -rn | grep tun | egrep '192|2001'
192.168.250.0/24   tun0               US         tun0
2001:470:XXXX:6464::/96           tun0           US         tun0
2001:470:XXXX:6464:ffff::1        tun0                          UHS        tun0

ping6 from the VPS:

Code:

# ping6 -c 3 2001:470:XXXX:6464::1.1.1.1
PING6(56=40+8+8 bytes) 2001:470:XXXX::1 --> 2001:470:XXXX:6464::101:101
16 bytes from 2001:470:XXXX:6464::101:101, icmp_seq=0 hlim=52 time=8.795 ms
16 bytes from 2001:470:XXXX:6464::101:101, icmp_seq=1 hlim=52 time=8.188 ms
16 bytes from 2001:470:XXXX:6464::101:101, icmp_seq=2 hlim=52 time=6.923 ms

--- 2001:470:XXXX:6464::1.1.1.1 ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 6.923/7.969/8.795/0.780 ms

After that is a matter of pointing my laptop to use this VPS as it's DNS/DNS64 resolver.

Code:

PS C:\Users\toor> nslookup twitter.com
Server:  dns64.vps-server.com
Address:  2001:470:XXXX:64::64

Non-authoritative answer:
Name:    twitter.com
Addresses:  2001:470:XXXX:6464::68f4:2ac1
          2001:470:XXXX:6464::68f4:2a41
          104.244.42.193
          104.244.42.65

PS C:\Users\toor>

PS C:\Users\toor> ping -n 3 twitter.com

Pinging twitter.com [2001:470:XXXX:6464::68f4:2ac1] with 32 bytes of data:
Reply from 2001:470:XXXX:6464::68f4:2ac1: time=50ms
Reply from 2001:470:XXXX:6464::68f4:2ac1: time=43ms
Reply from 2001:470:XXXX:6464::68f4:2ac1: time=42ms

Ping statistics for 2001:470:XXXX:6464::68f4:2ac1:
    Packets: Sent = 3, Received = 3, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 42ms, Maximum = 50ms, Average = 45ms
PS C:\Users\toor>

One point to note, I had to set the TCP MSS to 1440 as some IPv4-only websites won't load.

In /etc/sysctl.conf I added:

Code:

net.inet.tcp.mssdflt=1440

In pf.conf I updated the rule:

Code:

scrub in on $ext_if no-df random-id max-mss 1440

After this all was working as expected.

NAT64 with ipfw issues

Stardco

SirDice

Administrator

Stardco

Stardco

Stardco

olivier

mumu

subnetspider

SirDice

Administrator

mumu

mumu