PF PF port forwarding to jail running NSD on port 53

selflessscoundrel

selflessscoundrel

So, I'm probably overcomplicating things but the purpose of this exercise is as much functional as it is educational.

I set up a jail using ezjail for my nameserver and assigned it an IP of 192.168.0.4 on a local loopback interface lo1

The nsd daemon in the jail is running on port 53

I'm trying to have pf port forward port 53 from the external interface to the internal 192.168.0.4.

Internal drill commands on the server work, such as:
drill @192.168.0.4 ANY testdomain.lan

External drill commands to the server do not work:
drill @1.2.3.4 ANY testdomain.lan
Error: error sending query: Could not send or receive, because of network error

My pf.conf looks like this:
Code: 
#PUBLIC IP

IP_PUB="1.2.3.4"

IP_JAIL_NSD="192.168.0.4"


ext_if=vtnet0

jail_if="lo1"


NET_JAIL="192.168.0.0/29"


#PACKET NORMALIZATION

scrub in all


#Allow outbound connections from within the jails

nat pass on $ext_if from lo1:network to any -> (vtnet0)


#nameserver jail at 192.168.0.4

rdr pass on $ext_if proto {udp,tcp} from any to $IP_PUB port 53 -> $IP_JAIL_NSD port 53

Any theories? Do you need more information?
 
Can your jail actually contact anything on the outside at all? Can it, for example, ping your router (if you use any) or 8.8.8.8 for example? Because I somewhat doubt that, that nat rule looks somewhat peculiar to me (which is mostly because I seldomly append modifiers like that).

If it can then you have ruled out that the problem could reside with the outgoing NAT rule.

Also: from which location are you trying to reach the DNS server through the public IP address? If you're trying this on the server itself then that could also be affected by any other possible firewall rules or optional routing table entries, if you have any of course.
 
From within the jail I was able to perform a "host" and also was able to get a file from the internet using wget so I am able to connect to the outside from within the jail.

I have tried reaching port 53 from on the server, and also from outside the server to the public IP I haven't shared. Both fail. Using the internal jail IP of 192.168.0.4 it works fine.
 
Also of note in /etc/rc.conf of the base system (non jailed) to get the internal IP Addresses:

Code: 
cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"
 
Lamia said:
It appears your rule is missing "pass on $ext_if proto {udp,tcp} from any to $ext_if" and vice versa.
Click to expand...
So it's blocking the repsonses?

I ran tcpdump on the server and tried to drill from another server, see below for the error which is the server responding via UDP back to the requestor

tcpdump -nvvvpi vtnet0 udp and port domain

Code: 
17:52:18.839568 IP (tos 0x0, ttl 64, id 62148, offset 0, flags [none], proto UDP(17), length 174)
    1.2.3.4.53 > 5.6.7.8.44819: [bad udp cksum 0x405d -> 0x84a1!] 35772*-
 
I gotta get to sleep but I ran tcpdump on an external server while trying to run the drill command against the current pf config on my server. The tcpdump on the external server saw the outgoing UDP packets but did not see any return, so I'm 75% sure that I'm missing rules to allow the udp packets to go out from my server but I can't figure it out between the NAT and port forwarding. Any suggestions would be helpful.
 
If you are missing rule(s) to allow traffic out of your server, it would be "pass on $ext_if proto {udp,tcp} from $ext_if to any". That is why I said vice versa. You may ignore the "...from any to $ext_if..." if you already have it.

That issue is actually that of "pass on $ext_if...." It is responsible for allowing traffic through an interface in a specified direction.
 
Ok, more and more learning with PF and I think I am getting close, but I'm still not able to run a drill command from an external workstation to the server.

I am able to run a drill @internal_ip ANY domain.name successfully from the local server environment outside the jail but not a drill @externalip ANY domain.name


This is what the NAT and pass rules look like.

Note the SSH and IRC rules work fine and I have connectivity. It is just the port 53 connections that seem misconfigured.

Code: 
#PACKET NORMALIZATION
scrub in all


#Allow outbound connections from within the jails
nat pass on $ext_if from lo1:network to any -> (vtnet0)


#irc jail
rdr on $ext_if proto tcp from any to $IP_PUB port 6697 -> $IP_JAIL_IRC port 6697

#nameserver jail
rdr on $ext_if proto {udp,tcp} from any to $IP_PUB port 53 -> $IP_JAIL_NSD port 53


block in on $ext_if all
block out on $ext_if all
block in on $int_if all
block out on $int_if all

pass in on $ext_if proto tcp from any to $ext_if port 22

pass in on $ext_if proto tcp from any to $int_if port 6697


pass in on $ext_if proto {udp,tcp} from any to $ext_if port 53

pass in on $ext_if proto {udp,tcp} from any to $int_if port 53

pass in on $int_if proto {udp,tcp} from any to $int_if port 53

pass out on $ext_if proto {udp,tcp} to any port 53

pass out on $int_if proto {udp,tcp} to any port 53
 
Code: 
#PUBLIC IP
IP_PUB="1.2.3.4" ##obfuscated
IP_JAIL_IRC="192.168.0.2"
IP_JAIL_NSD="192.168.0.4"

ext_if=vtnet0
int_if="lo1"
jail_if="lo1"  #unnecessary duplication I know

NET_JAIL="192.168.0.0/29"  #not used

#PACKET NORMALIZATION
scrub in all


#Allow outbound connections from within the jails
nat pass on $ext_if from lo1:network to any -> (vtnet0)


#irc jail at 192.168.0.2
rdr on $ext_if proto tcp from any to $IP_PUB port 6697 -> $IP_JAIL_IRC port 6697

#nameserver jail at 192.168.0.4
rdr on $ext_if proto {udp,tcp} from any to $IP_PUB port 53 -> $IP_JAIL_NSD port 53

block log in on $ext_if all
block log out on $ext_if all
block log in on $int_if all
block log out on $int_if all

#pass out quick inet keep state

pass in on $ext_if proto tcp from any to $ext_if port 22

pass in on $ext_if proto tcp from any to $int_if port 6697

pass in on $ext_if proto {udp,tcp} from any to $ext_if port 53

pass in on $ext_if proto {udp,tcp} from any to $int_if port 53

pass in on $int_if proto {udp,tcp} from any to $int_if port 53

pass out on $ext_if proto {udp,tcp} to any port 53

pass out on $int_if proto {udp,tcp} to any port 53
 
At this point I think it is an NSD configuration issue.

I installed NSD onto the main server to remove all consideration of NAT issues. Then I disabled pf all together. I am still having issues, so I will move on to debugging NSD.

Thanks
 
The final update. My cloud server is fine. It's my ISP blocking responses apparantly. I tested other nameservers on other domains - nslookup and drill commands don't work locally on my machine but they work on my cloud server.

Thank you all, and sorry for dragging you all through it.
 
selflessscoundrel said:
Thank you all, and sorry for dragging you all through it.
Click to expand...
No, thank you for the update. Most of us are here because we enjoy helping by solving puzzles and trying to come up with solutions to problems, so it's always nice if someone also takes the time to share what the actual problem(s) turned out to be.
 
I have found the solution and I'll post the rules I found here:

PF.CONF
Code: 
#PUBLIC IP
IP_PUB="External IP"
IP_JAIL_NSD="Internal IP"

ext_if=vtnet0  #check your ifconfig to see the name of your external interface
int_if="lo1"   #created in rc.conf

#PACKET NORMALIZATION
scrub in all

#Allow outbound connections from within the jail
nat pass on $ext_if from lo1:network to any -> (vtnet0)

#nameserver jail port forwarding
rdr on $ext_if proto {udp,tcp} from any to $IP_PUB port 53 -> $IP_JAIL_NSD port 53

#Edit - I haven't worked out all the issues with the below settings, any they shouldn't be needed unless for additional hardening.
#Block all traffic by default
#block in log on $ext_if all
#block out log on $ext_if all
#block in log on $int_if all
#block out log on $int_if all

#allow SSH connection to the main server
#pass in on $ext_if proto tcp from any to $ext_if port 22

#allow DOMAIN connections coming into to the external interface from anywhere pass into the internal interface
#pass in on $ext_if proto {udp,tcp} from any to $int_if port 53

Also of note, since I used jails and created a loopback interface to assign internal IPs to the jails this is my rc.conf:

RC.CONF snippet

Code: 
ezjail_enable="YES"
cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"
pf_enable="YES"

lo1 is distinct from the default lo0 loopback interface at 127.0.0.1.


This configuration will not allow you to drill @localIP ANY yourdomain.com from the nameserver itself, it will fail. You need to test it from an outside server into your nameserver, and don't make the same mistake I did. Ensure your local internet provider allows you to run nslookup or drill commands to specified servers! Otherwise you'll waste everyone's time, haha. Sorry again.
 
You must log in or register to reply here.
Back
Top