Solved pf NAT

LSD

LSD

I have a computer running FreeBSD that I want to use as a gateway between my LAN and the internet.
Here is a diagram:

(internet) | | [ ISP ROUTER ] [ 192.168.0.1 ] | | | | [ ue0 192.168.0.106 ] [ FreeBSD ] [ ue1 10.0.0.0 ] | | | | [ fxp0 10.0.0.1 ] [ OpenBSD (irrelevant, could be a switch, router, xbox/ps4 etc) ]

Here is the output from FreeBSD:

% ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
ether b8:27:eb:a9:5f:e8
inet 192.168.0.106 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ue1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 00:e0:4c:36:36:f0
inet 10.0.0.0 netmask 0xff000000 broadcast 10.255.255.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
groups: pflog


% cat pf.conf
# external interface, internet
ext_if = "ue0"

# internal interface, LAN
int_if = "ue1"
localnet = $int_if:network"

match out on $ext_if inet from $localnet nat-to ($ext_if)

pass from { self, $localnet }


% pfctl -nf pf.conf
pfctl -nf pf.conf
pf.conf:13: syntax error
no IP address found for ue1:network"
pf.conf:16: could not parse host specification


I want openbsd to be able to connect to the internet through freebsd
I am reading through `The Book of PF' which is where I got that pf.conf from
I don't care about security right now (default deny etc) I just want openbsd to connect to the internet through freebsd
I shouldn't need to configure openbsd, since I want to be able to replace it with a regular switch or router or game console etc

Thanks for any help
 
The most obvious issue is the actual syntax error. You have a double quote at the end of $int_if:network".
Either surround the whole value in double quotes, like the other settings, or don't have a random double quote at one end.

Just for interest, on my test bhyve system, I literally just have the following for nat support. (With re0 being the external interface and 192.168.8.0/24 being the nat network)

Code: 
nat on re0 from {192.168.8.0/24} to any -> (re0)

Edit: looks like the following also works which auto configures the nat network

Code: 
nat on re0 from {re1:network} to any -> (re0)

Note you also need gateway_enable="yes" in /etc/rc.conf for FreeBSD to forward packets between interfaces.
 
The next problem is that there's no match keyword in Freebsd pf(4).

I love Hansteen's work, but keep in mind that he covers at least three different Pf versions. The Openbsd root version and its Freebsd and Netbsd ports have significant differences. He usually highlights which version he's writing about pretty explicitly, but I'm guessing you missed it.

Consider "The Book of Pf" to be advanced, and start with the Pf section of the handbook. There's a handy section called "A Simple Gateway with NAT".

getopt said:
PF is just a packet filter. What you need is to setup bridge(4) first. Then you can write PF-rules on interface bridge if you like.
Click to expand...
This would only be necessary if Openbsd is running in a jail or virtual machine, no? I didn't see where this was specified.
 
Double NAT is in most cases a very bad idea and will cause more issues than it solves, what are you trying to accomplish in the end?
 
diizzy said:
Double NAT is in most cases a very bad idea and will cause more issues than it solves, what are you trying to accomplish in the end?
Click to expand...
Basically I want FreeBSD to be able to `give' internet to whatever I plug into it.

The computer running FreeBSD has 2 ethernet ports, ue0 and ue1, ue0 gets internet and I want ue1 to give out internet, ether to a computer or game console directly or a switch after filtering the traffic through a firewall.

OpenBSD's fxp0 interface should send a packet to FreeBSD's ue1 interface (they are connected via an ethernet cable), then ue1 should forward that traffic to ue0 (they are both on the same machine). When ue0 gets a reply it should forward it to ue1, which would send it to fxp0, and thus OpenBSD should have internet access.
That is how I assume this should work, routers/switches do it automatically.

I could just use a switch or router and it would work through plug and play, but I want to use FreeBSD so I can use PF to filter traffic in between.

I thought this was a very simple thing, but I haven't had any luck with it.
 
usdmatt said:
The most obvious issue is the actual syntax error. You have a double quote at the end of $int_if:network".
Either surround the whole value in double quotes, like the other settings, or don't have a random double quote at one end.

Just for interest, on my test bhyve system, I literally just have the following for nat support. (With re0 being the external interface and 192.168.8.0/24 being the nat network)

Code: 
nat on re0 from {192.168.8.0/24} to any -> (re0)

Edit: looks like the following also works which auto configures the nat network

Code: 
nat on re0 from {re1:network} to any -> (re0)

Note you also need gateway_enable="yes" in /etc/rc.conf for FreeBSD to forward packets between interfaces.
Click to expand...

I replaced that line I had in pf.conf with yours, and also added the lines needed to rc.conf and sysctl.conf (net.inet.ip.forwarding=1)
Rebooted both FreeBSD and OpenBSD, but still not working.

From FreeBSD(10.0.0.0 ue1 and 192.168.0.106 ue0) I can ping OpenBSD (10.0.0.1)
OpenBSD can ping 10.0.0.0 and 192.168.0.106 but not www.google.com, not 192.168.0.104 (another computer on the network) or 192.168.0.1, it doesn't have internet access.


I'm ready to just give up and buy a router that has a built in firewall I can customize or something, so it will `just work' through plug and play.
 
% cat pf.conf
# external interface, internet
ext_if = "ue0"

# internal interface, LAN
int_if = "ue1"

nat on $ext_if inet from $int_if:network to any -> ($ext_if)
nat on $ext_if from {$int_if:network} to any -> ($ext_if)

pass in on $int_if from $int_if:network to $ext_if:network
pass out on $ext_if from $int_if:network to $ext_if:network

pass from { self, $int_if:network }

% sudo pfctl -sr
pass inet from 127.0.0.1 to any flags S/SA keep state
pass inet from 192.168.0.106 to any flags S/SA keep state
pass inet from 10.0.0.0/8 to any flags S/SA keep state
pass inet6 from ::1 to any flags S/SA keep state
pass on lo0 inet6 from fe80::1 to any flags S/SA keep state

Does that look right?

Thanks for any help, it is very much appreciated.
 
I just noticed that you're using 10.0.0.0 as the IP address for ue1. This is the zero address for subnet 10.0.0.0/8. Try using 10.0.0.1 for ue1 and 10.0.0.2 for Openbsd.

Keep in mind that in Pf, the last rule that matches wins, so:
Code: 
nat on $ext_if inet from $int_if:network to any -> ($ext_if)
nat on $ext_if from {$int_if:network} to any -> ($ext_if)
# Why two NAT rules? They're equivalent. The first one does nothing

pass in on $int_if from $int_if:network to $ext_if:network
pass out on $ext_if from $int_if:network to $ext_if:network
# These two rules do nothing because...

pass from { self, $int_if:network }
#...this rule matches a superset of the traffic that the previous rules would match

% sudo pfctl -sr
pass inet from 127.0.0.1 to any flags S/SA keep state
# Previous must be a default rule. I don't see it in your file
pass inet from 192.168.0.106 to any flags S/SA keep state
# Previous is from "pass self"
pass inet from 10.0.0.0/8 to any flags S/SA keep state
# Previous is from "pass $int_if:network"
LSD said:
Does that look right?
Click to expand...
I think using the zero address is the main problem here, though. Those rules should work.
 
Jose said:
I just noticed that you're using 10.0.0.0 as the IP address for ue1. This is the zero address for subnet 10.0.0.0/8. Try using 10.0.0.1 for ue1 and 10.0.0.2 for Openbsd.

Keep in mind that in Pf, the last rule that matches wins, so:
Code: 
nat on $ext_if inet from $int_if:network to any -> ($ext_if)
nat on $ext_if from {$int_if:network} to any -> ($ext_if)
# Why two NAT rules? They're equivalent. The first one does nothing

pass in on $int_if from $int_if:network to $ext_if:network
pass out on $ext_if from $int_if:network to $ext_if:network
# These two rules do nothing because...

pass from { self, $int_if:network }
#...this rule matches a superset of the traffic that the previous rules would match

% sudo pfctl -sr
pass inet from 127.0.0.1 to any flags S/SA keep state
# Previous must be a default rule. I don't see it in your file
pass inet from 192.168.0.106 to any flags S/SA keep state
# Previous is from "pass self"
pass inet from 10.0.0.0/8 to any flags S/SA keep state
# Previous is from "pass $int_if:network"

I think using the zero address is the main problem here, though. Those rules should work.
Click to expand...

I tried changing the addresses and running that new pf.conf, but still no luck.

OpenBSD can still only ping 10.0.0.1 and 192.168.0.106, nothing else.
I have also tried plugging the ethernet cable from ue1 into a Windows 7 laptop, it couldn't ping anything at all until I manually added 10.0.0.1 as the gateway, then it could only ping 10.0.0.1 and 192.168.0.106
Is this maybe a hint to the problem? Since shouldn't that stuff automatically be setup when I connect to a router?

This FreeBSD is a fresh install, I have only added
gateway_enable="yes" -> rc.conf
net.inet.ip.forwarding=1 - >sysctl.conf
and pf is running the above new pf.conf
Does anything else need to be setup like dhcp?


anonymous9 said:
[...]
I think that's all you want. NAT + screened LAN.
Click to expand...
Yes that is all I want, I don't know why it is so difficult to setup
 
It isn't difficult to set up.

I would scrap everything other than the nat rule to start off. Not that you have any drop rules or a complex ruleset but I never understand why people start setting up firewall rules before something is actually working.

You also don't need the forwarding sysctl. That is set by having gateway_enable set.

Put the FreeBSD machine on 10.0.0.1 and the OpenBSD box on 10.0.0.2. Add a single pf rule as below and show the output of the given commands (After restarting pf). Check ping from the OpenBSD box and whether it can see the ISP router.

Code: 
nat on ue0 from {ue1:network} to any -> (ue0)

# ifconfig -a
# pfctl -sn
# netstat -rn


As I said above I'm using just this single rule on a bhyve host and all guests have no issue getting out via my lan router. All I did was add this rule and enable pf & gateway in rc.conf. It took about 13 seconds.

Plugging a laptop into ue1 will not get an address as there is no dhcp. If you want to be able to just plug random devices into ue1 (or a switch connected to it) and get an address automatically, you'll want to run something like dnsmasq to provide dhcp but that can be set up once nat is actually working. Again, it's not worth running before you can walk. Get nat working, then worry about dhcp and firewall rules.
 
usdmatt said:
It isn't difficult to set up.

I would scrap everything other than the nat rule to start off. Not that you have any drop rules or a complex ruleset but I never understand why people start setting up firewall rules before something is actually working.

You also don't need the forwarding sysctl. That is set by having gateway_enable set.

Put the FreeBSD machine on 10.0.0.1 and the OpenBSD box on 10.0.0.2. Add a single pf rule as below and show the output of the given commands (After restarting pf). Check ping from the OpenBSD box and whether it can see the ISP router.

Code: 
nat on ue0 from {ue1:network} to any -> (ue0)

# ifconfig -a
# pfctl -sn
# netstat -rn


As I said above I'm using just this single rule on a bhyve host and all guests have no issue getting out via my lan router. All I did was add this rule and enable pf & gateway in rc.conf. It took about 13 seconds.

Plugging a laptop into ue1 will not get an address as there is no dhcp. If you want to be able to just plug random devices into ue1 (or a switch connected to it) and get an address automatically, you'll want to run something like dnsmasq to provide dhcp but that can be set up once nat is actually working. Again, it's not worth running before you can walk. Get nat working, then worry about dhcp and firewall rules.
Click to expand...


FreeBSD:

fbsd% ifconfig -a
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
ether b8:27:eb:a9:5f:e8
inet 192.168.0.106 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ue1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 00:e0:4c:36:36:f0
inet 10.0.0.1 netmask 0xff000000 broadcast 10.255.255.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33160
groups: pflog

fbsd% sudo pfctl -sn
nat on ue0 inet from 10.0.0.0/8 to any -> (ue0) round-robin

fbsd% netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.0.1 UGS ue0
10.0.0.0/8 link#3 U ue1
10.0.0.1 link#3 UHS lo0
127.0.0.1 link#1 UH lo0
192.168.0.0/24 link#2 U ue0
192.168.0.106 link#2 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#1 UH lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#1 U lo0
fe80::1%lo0 link#1 UHS lo0
ff02::/16 ::1 UGRS lo0


FreeBSD can't ping 10.0.0.2 anymore and OpenBSD can't ping 10.0.0.1 or 192.168.0.106 ether

fbsd% ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available


If I try to ping 10.0.0.1 on OpenBSD it says `no route to host'
Then when I run `route add default 10.0.0.1'
and try to ping 10.0.0.1 (FreeBSD) again
it says `host is down', even though computers on 192.168.0/24 can ping it's 192.168.0.106 address just fine
 
FreeBSD can't ping 10.0.0.2 anymore

ping: sendto: No buffer space available
Click to expand...

The error suggests something a bit more obscure than just configuration problems. Looking online suggests possible dodgy connections/cables as I doubt you're actually running out of buffer memory. A reboot may solve it temporarily.

If I try to ping 10.0.0.1 on OpenBSD it says `no route to host'
Click to expand...

You should not get a 'no route to host' error on OpenBSD when pinging 10.0.0.1. This suggests a basic configuration error on the OpenBSD machine. A default route is not needed just to ping a device on the same network range.
 
Interesting question... is /etc/rc.conf case sensitive with YES and NO and NONE?
Click to expand...

No, these are the patterns it uses (so basically no,false,off,0 & yes,true,on,1 all work)
I tend to put all mine in lowercase as I think it looks neater.

Code: 
[Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|[Oo][Nn]|1
[Nn][Oo]|[Ff][Aa][Ll][Ss][Ee]|[Oo][Ff][Ff]|0

None is a special case used by Sendmail* which basically just switches 4 settings off: (again not case-sensitive)

Code: 
case ${sendmail_enable} in
[Nn][Oo][Nn][Ee])
        sendmail_enable="NO"
        sendmail_submit_enable="NO"
        sendmail_outbound_enable="NO"
        sendmail_msp_queue_enable="NO"
        ;;
esac

*Possible other services may support "none", but they too would need code in their own rc script to handle it.
 
LSD said:
Is this maybe a hint to the problem? Since shouldn't that stuff automatically be setup when I connect to a router?
Click to expand...
Not unless you set up DHCP on the Freebsd host.

LSD said:
Does anything else need to be setup like dhcp?
Click to expand...
Yes. Look at Installing and Configuring a DHCP Server.

What is the default gateway on the Freebsd machine? ( netstat -nr) On the Openbsd machine? (same command)

LSD said:
Yes that is all I want, I don't know why it is so difficult to setup
Click to expand...
I suggest you learn the basics of TCP/IP networking. I learned them by reading the Crab Book, but that may be out of date. Maybe someone else can recommend something more modern.
 
I finally got it working!!!

anonymous9 said:
[...]

Show your FreeBSD /etc/pf.conf and /etc/rc.conf files. And on OpenBSD your /etc/hostname.ue1 (or whatever interface it is) and /etc/mygate.

[...]
Click to expand...

On OpenBSD, I added:
inet 10.0.0.2/24
to /etc/hostname.fxp0

and created a /etc/mygate with `10.0.0.1'
then I rebooted and everything works fine!
I can ping 192.168.0.1, www.google.com, 192.168.0.104 etc

Prior to that, I would just run these commands after logging in:
# ifconfig fxp0 inet 10.0.0.2/24
and
# route add default 10.0.0.1
Having those lines in the files makes all the difference apparently

On FreeBSD I also added
ifconfig_ue1="inet 10.0.0.1/24"
to rc.conf

pf.conf has only:
nat on ue0 from {ue1:network} to any -> (ue0)

Thank you to everyone who helped me, this was a very annoying problem I have been trying to fix.
 
You must log in or register to reply here.
Back
Top