PF To NAT or not to NAT? A host machine with PF and a jail running mail services

K

kalleboy

Greetings. I have public NIC with few public IPs assigned;

$ext_if = my external NIC with my public Internet addresses
$public_IP_1 = one of my public Internet IP (assigned as an alias and working on $ext_if)
And 10.10.10.2 is my jail running on FreeBSD 14 host machine.

I have the following PF rules to redirect Internet ports of the mail services, to one of my Jail;

Code: 
rdr on $ext_if inet proto tcp from any to $public_IP_1 port {smtp, submission, smtps, imap, imaps, pop3, pop3s} -> 10.10.10.2
pass in on $ext_if proto tcp from any to 10.10.10.2 port {smtp, submission, smtps, imap, imaps, pop3, pop3s}

So far, they seem to be working. For instance, a connection to My.Public.IP port 465, 110 goes to the same port running in Jail.

However, my jail cannot reach outside, cannot ping to a public IP address, cannot connect to a public website. How to allow these without adding a general NAT rule? Like:
nat on $ext_if from ($int_if:network) to any -> ($ext_if:0)

With such a rule above, I have no problem about the outside connection of the jails, but I feel like my jails ($int_if) are too *public* and not restrictive.

Is it mandatory to set up such a NAT rule in PF in my case?
Is there any way to allow jails to do ping, traceroute, connecting to 80 ports of public servers, without NAT?

A rule like this did not work;
pass out on $ext_if proto icmp from 10.10.10.2 to any

Any clue would be much appreciated - many thanks in advance.
 
kalleboy said:
However, my jail cannot reach outside, cannot ping to a public IP address, cannot connect to a public website.
Click to expand...
Think of it this way, does your mail server actually need to do any of this?
kalleboy said:
How to allow these without adding a general NAT rule?
Click to expand...
You don't, it will require some sort of NAT rule.
Code: 
nat on $ext_if from ($int_if:network) to any -> ($ext_if:0)
It doesn't need to allow the entire subnet, you can actually limit this to only the host.
Code: 
nat on $ext_if from 10.10.10.2 to any -> ($ext_if:0)

kalleboy said:
With such a rule above, I have no problem about the outside connection of the jails, but I feel like my jails ($int_if) are too *public* and not restrictive.
Click to expand...
Add rules to limit the outgoing traffic.
 
Hi SirDice thanks a lot for your truly great suggestions - as always!

SirDice said:
Think of it this way, does your mail server actually need to do any of this?
Click to expand...
Actually in my case, my mail server will need; rspamd connecting to outside spam rbl servers, ClamAV connecting to outside fetching updated DBs, and overall pkg updates, upgrades, and so on.

According to your suggestions, I've started writing my pf.conf from scratch, and the result is here (thinking I'm going to order one more public IP from my server provider for Web server VM);

Bash: 
ext_if="re0"
ext_publicweb_ip="163.19.38.192"
ext_publicmail_ip="163.19.38.193"
vms_network="{10.10.10.0/29}"
mail_vm="10.10.10.2"
web_vm="10.10.10.3"

nonrout = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 169.254.0.0/16, 192.0.2.0/24, 198.18.0.0/15, 0.0.0.0/8, 240.0.0.0/4, 255.255.255.255/32 }"
icmp_types = "{ echoreq, unreach, timex }"
icmp6_types = "{ echoreq, unreach, timex, toobig, paramprob, neighbrsol }"

# Blacklist - Whitelist
table <whitelist> persist file "/var/pf/whitelist.txt"
table <blocklist> persist file "/var/pf/blacklist.txt"

set skip on lo0
set state-policy if-bound
set loginterface $ext_if
set ruleset-optimization basic
set limit { states 200000, frags 200000, src-nodes 100000, table-entries 400000 }

scrub in on $ext_if all fragment reassemble

nat on $ext_if from {$vms_network} to any -> ($ext_if:0)

#Redirection to Mail VM
rdr on $ext_if inet proto tcp from any to $ext_publicmail_ip port {smtp, submission, smtps, imap, imaps, pop3, pop3s} -> $mail_vm

#Redirection to Web VM
rdr on $ext_if inet proto tcp from any to $ext_publicweb_ip port {www, https} -> $web_vm

#Antispoof
antispoof quick for $ext_if inet
antispoof quick for ($ext_if) inet6

#Whitelisted IPs
pass quick on $ext_if from <whitelist> to any flags any keep state

#Blacklisted IPs
block in quick on $ext_if from <blocklist>
block out quick on $ext_if to <blocklist>
block return in log all
block in quick on $ext_if from $nonrout to any
block out quick on $ext_if from any to $nonrout
block proto udp
block in quick from no-route to any
block in quick from urpf-failed to any

#Pass for Mail VM
pass in on $ext_if proto tcp from any to $mail_vm port {smtp, submission, smtps, imap, imaps, pop3, pop3s} flags S/SA synproxy state
pass proto tcp from $mail_vm to any port {smtp, smtps, submission} flags S/SA synproxy state

#Pass for Web VM
pass in on $ext_if proto tcp from any to $web_vm port {www, https} flags S/SA synproxy state

#Passes for the main Host machine: Ssh, local unbound, ntp, ping, dhcp access and traceroute
pass in quick on $ext_if proto { tcp udp } to ($ext_if) port {domain, ntp}
pass quick inet proto icmp icmp-type $icmp_types
pass quick inet6 proto ipv6-icmp icmp6-type $icmp6_types
pass in quick on $ext_if proto tcp from any to ($ext_if) port 22 flags S/SA synproxy state
pass quick on $ext_if proto udp from any to ($ext_if) port = 67 keep state
pass out quick on $ext_if inet proto udp from ($ext_if) to any port 33433 >< 33626 keep state

Do you think, overall, the configuration above is fine? Anything weird there you might wish to point? Any suggestions would be much, much appreciated!

Thanks in advance, one more time!

Regards.
 
Last edited:
While we're at it, the questions I wonder and want to ask; is there any limit on number of VMs that would be NAT'd on host system? I guess once I read something like the system would be out of free ports, if there're too many VMs and using NAT?
 
kalleboy said:
I guess once I read something like the system would be out of free ports, if there're too many VMs and using NAT?
Click to expand...
Doesn't have anything to do with the number of VMs, it has to do with the number of (concurrent) connections. For every connection NAT needs to pick a random source port (state table keeps track of these) and there are only a finite number of ports (65535) to pick from. More VMs (or LAN clients, same problem) usually means more concurrent connections too. So you're going to run out of ports quicker.
 
Oh, now I got it better.. So it all depends on concurrent connections.

And lastly, SirDice do you think my PF rules above is all set according to one host + 2 VMs?

I'd be grateful and it'd help a lot if you could give a feedback on my rules.

Thanks!
 
You must log in or register to reply here.
Back
Top