Solved rules blocking all IN traffic from LAN to jails

unknownuser · Feb 19, 2020

Hello everyone,

I have a FreeBSD host with 3 Jails using VNET and EPAIR, where each jail has got its own subnet, and than PF nats the traffic.
The issue came up when I introduced the block in policy, effectively restricting all Inbound traffic to the host.

What I need is to allow icmp, tcp, udp on 80 and 443 from LAN to the Jails. I have been trying countless of syntaxes, mostly random examples at this point, hoping that one would work. I read the man page, but still can't understand what I need and how to write it.

FreeBSD's host: 10.16.0.101 vtnet0
Jails:
node1: 192.168.69.254 epair0
node2: 192.168.68.254 epair1
fwlb: 192.168.0.254 epair2

/etc/pf.conf

Code:

ext_if="vtnet0"
NET_JAIL="{ 192.168.69.0/24 192.168.68.0/24 192.168.0.0/24 }"

jails_if="{ epair0a epair1a epair2a }"

scrub in all

# Won't filter traffic on loopback
set skip on lo0

# nat all jail traffic
nat on $ext_if inet from any to any -> ($ext_if)

# Protects against activity from spoofed or forged IPs
antispoof for $ext_if inet

# Exposing JAIL ports to the HOST
#rdr pass on $ext_if inet proto tcp to port {80, 443} -> $NET_JAIL

# Allow SSH/d
pass in quick on $ext_if proto tcp from any to any port ssh

# Allow traceroute
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state

# Allow ICMP ping
pass inet proto icmp from any to any

# Allow all OUT, block all IN on $ext_if
block in on $ext_if
pass out

Thanks

VladiBG · Feb 20, 2020

look at this line
ext_if="vtnet0"
...
block in on $ext_if

Code:

     For each packet processed by the packet filter, the filter    rules are
     evaluated in sequential order, from first to last.     The last matching
     rule decides what action is taken.     If no rule matches the    packet,    the
     default action is to pass the packet.

p.s.
Your rdr rule is not correct. Check the pf.conf (5) for examples

unknownuser · Feb 20, 2020

VladiBG said:
look at this line
ext_if="vtnet0"
...
block in on $ext_if

Code:

For each packet processed by the packet filter, the filter rules are evaluated in sequential order, from first to last. The last matching rule decides what action is taken. If no rule matches the packet, the default action is to pass the packet.

p.s.
Your rdr rule is not correct. Check the pf.conf (5) for examples

Even if I remove the $ext_in from block in and I leave block in alone, incoming traffic from LAN to the jails still gets blocked. I just can't figure how to tell PF to allow incoming traffic from LAN to the jails.

`Orum · Feb 20, 2020

unknownuser said:
Even if I remove the $ext_in from block in and I leave block in alone, incoming traffic from LAN to the jails still gets blocked. I just can't figure how to tell PF to allow incoming traffic from LAN to the jails.

So you're just using "block in"? That will just block all incoming traffic (except for those in the state table created by outgoing connections).

What he was saying in a not-so-subtle way is that PF is, by default, a last match firewall. That is, the last rule that matches the traffic flowing through the ruleset will be the one that is applied, not the first one.

You have two options: Either move your block rule to the top, or make all your rules "quick", i.e. first match. I prefer the latter as it (allegedly) is better for performance, but I haven't ever been in an environment that can truly test that claim. It also makes the rules easier for a human to parse and harder to screw up, IMHO.

You'll also need to uncomment and adjust your rdr rule so that it redirects to an IP address (instead of network addresses), if you want daemons to listen for traffic within the jail(s).

unknownuser · Feb 20, 2020

`Orum said:
So you're just using "block in"? That will just block all incoming traffic (except for those in the state table created by outgoing connections).

What he was saying in a not-so-subtle way is that PF is, by default, a last match firewall. That is, the last rule that matches the traffic flowing through the ruleset will be the one that is applied, not the first one.

You have two options: Either move your block rule to the top, or make all your rules "quick", i.e. first match. I prefer the latter as it (allegedly) is better for performance, but I haven't ever been in an environment that can truly test that claim. It also makes the rules easier for a human to parse and harder to screw up, IMHO.

You'll also need to uncomment and adjust your rdr rule so that it redirects to an IP address (instead of network addresses), if you want daemons to listen for traffic within the jail(s).

Oh got it now. Thanks. So I did a little bit of re-arranging and perhaps a bit of polishing as well, and the jails are now reachable from within LAN. However, it seems like jails outgoing DNS requests are being blocked somehow, even if I specified pass out all.

This is how my /etc/pf.conf looks like now

Code:

ext_if="vtnet0"
NET_JAIL="{ 192.168.69.0/24 192.168.68.0/24 192.168.0.0/24 }"

scrub in all

# Won't filter traffic on loopback
set skip on lo0

# nat all jail traffic
nat on $ext_if inet from $NET_JAIL to any -> ($ext_if)


# Allow all OUT, block all IN on $ext_if
block in
pass out all

# Protects against activity from spoofed or forged IPs
antispoof for $ext_if inet

# Allowing inbound traffic to jails
pass quick proto tcp from any to $NET_JAIL port { 443, 80, 53 } flags S/SA keep state

# Allow traceroute
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state

# Allowing outbound DNS traffic (doesn't work inside jails??)
pass out proto { tcp udp } from any to any port domain

# Allow ICMP ping
pass inet proto icmp from any to any

# Allow SSH/d
pass in quick on $ext_if proto tcp from any to any port ssh flags S/SA keep state

As you can see I have added pass out proto { tcp udp } from any to any port domain, but still the jails cannot ping any internal nor external domain.
The DNS server is 172.12.0.1. It all works if I disable PF or I enable all IN and OUT traffic.

Not sure what I am doing wrong really. I fix something and I end up screwing something else it seems.

genneko · Feb 20, 2020

Hello,

unknownuser said:
As you can see I have added pass out proto { tcp udp } from any to any port domain, but still the jails cannot ping any internal nor external domain.

From the host's point of view, packets coming from the jails are inbound at first. So they are blocked by the block in rule and they are never forwarded nor NATed. You need a rule like pass from $JAIL_NET.

BTW, I wonder if you really need the nat rule for jails-to-LAN traffic because for the opposite direction the jails seem directly reachable from the LAN without any rdr rule.

unknownuser · Feb 20, 2020

genneko said:
Hello,

From the host's point of view, packets coming from the jails are inbound at first. So they are blocked by the block in rule and they are never forwarded nor NATed. You need a rule like pass from $JAIL_NET.

BTW, I wonder if you really need the nat rule for jails-to-LAN traffic because for the opposite direction the jails seem directly reachable from the LAN without any rdr rule.

Oh didn't know that from the host point of view outbound traffic inside the jail is actually seen as inbound. That explains why ...

Anyways, this rule did it

 

pass in proto { tcp udp } from $NET_JAIL to any port domain

Thanks as always

genneko · Feb 20, 2020

unknownuser said:
Oh didn't know that from the host point of view outbound traffic inside the jail is actually seen as inbound.

I think so at least for VNET jails as they have their own network stack and can be regarded as separate entities from the host.
I'm not sure about non-VNET jails which share the host's IP addresses, but I usually use pass from $JAILNET for them too.