Solved PF Port Range For Jails Misdirecting Host Requests

Hello,

I have an interesting misconfiguration that I could use some insight on. I have a 'turn' service inside of a jail and used a PF rule to send any ports within a certain range to that jail. Unfortunately, this appears to have an impact on the jail host performing certain actions like portsnap fetch.

Is there anyway to update the below pf.conf rules to allow for the host to use those external ports if it generates the request? Or am I thinking about this wrong? I'll apologize in advance for any misuse of terminology. I still feel like a BSD noob.
Code:
# Public IP address
IP_PUB="192.168.50.104"

# Packet normalization
scrub in all

# Allow outbound connections from within the jails
nat on igb0 from lo1:network to any -> (igb0)

# turn jail at lo1:192.168.0.7
rdr on igb0 proto { tcp udp } from any to $IP_PUB port 49152:65535 -> 192.168.0.7 port 49152:65535
 
I have a 'turn' service inside of a jail
What service is that? What does it do?
Unfortunately, this appears to have an impact on the jail host performing certain actions like portsnap fetch.
That's outgoing, it shouldn't have anything to do with any redirections on incoming connections.
Or am I thinking about this wrong?
At first glance there doesn't appear to be anything wrong with your rules. The redirections look somewhat excessive but it shouldn't be a problem.

What are the exact errors you're getting with portsnap fetch?
 
The turn service is the application net/coturn. It is supposedly a requirement for the Matrix VOIP features. See below why I ruled it out.

I agree. It doesn't make any sense, but if I restart pf with that last line commented out, portsnap works without an error. It is quite puzzling.

Commented out:

# portsnap auto
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from metapeer.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Aug 12 11:09:34 PDT 2019 to Mon Aug 12 11:38:04 PDT 2019.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 4 patches.
(4/4) 100.00% done.
done.
Applying patches...
done.
Fetching 0 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/net-mgmt/statsd/
/usr/ports/textproc/kibana5-search-guard/
/usr/ports/textproc/kibana5/
/usr/ports/www/yarn/
Building new INDEX files... done.


Entry in PF results in a partial failure. This is the same effect if I turn the jail off and leave the entry in place in the PF. Sometimes it works, but 90% of the time it will fail to fetch

# portsnap auto
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from metapeer.portsnap.freebsd.org... failed.
Fetching snapshot tag from your-org.portsnap.freebsd.org... failed.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Latest snapshot on server matches what we already have.
No updates needed.
Ports tree is already up to date.


Sometimes it will fail all possible mirrors. I originally thought it was a DNS issue (pings would fail due to no entries in DNS), but no other machines had this issue and it's only an issue when I have that line running in the PF.
 
My basic understanding is that the last line with the large port range is resulting in the following:

1. Make request to external server.
2. Connection established with port in the below range of 49152:65535 specified.
3. Traffic is misdirected to the jail. (DOH!)

I've updated the ruleset to comment out the last line and added the first two entries to see if that will fix the problem. I'll post an update if that fixed it.


# turn jail at 192.168.0.7
rdr on igb0 proto { tcp udp } from any to $IP_PUB port 3478 -> 192.168.0.7 port 3478
rdr on igb0 proto { tcp udp } from any to $IP_PUB port 5349 -> 192.168.0.7 port 5349
rdr on igb0 proto { tcp udp } from any to $IP_PUB port 49152:65535 -> 192.168.0.7 port 49152:65535
 
No, when the connection is made there's a new dynamic state being created. It's this state that allows the return traffic. The redirections only work for connections originating from the other side.
 
Regardless of the ultimate reasoning, having those large amount of non-standard ports to redirect in pf.conf was causing issues with DNS. This was with or without the jail being turned on, so either my configuration was wrong or there is a bug in pf. Solved because they were unnecessary and turn works without those ports set to redirect.

If this is a bug, I'll happily work to recreate it in an alternate system and submit a bug report, but I'd need some guidance on the process.
 
Back
Top