We have a PF firewall with NAT running on FreeBSD-12 setup as a gateway router. We are attempting to transition from IPTables on CentOS to PF on FreeBSD. However, we have encountered a serious problem with a third-party ssh client running on MS Windows. This software randomly drops its connection with the upstream VAN provider that mandates the use of this software. This problem only began following the move to the FreeBSD PF gateway and so suspicion falls mainly there. We have discovered that PF is blocking some packets from and to the VAN despite there being specific allowances for such traffic in the rule set.
I do not understand TCP to any great extent so I cannot decipher what is causing PF to ignore the specific PF rules provided for the VAN. I can guess that there is something about the tcp flags or timeouts but I cannot interpret the traces. I tried simply removing the Calomel macros and just go with nothing on the rules, which should have given me '[FONT=courier new]keep state flags S/SA[/FONT]' . But that did not seem to work.
Switching to explicitly setting 'keep state flags S/SA' appears to eliminate the problem but I do not understand why this is necessary.
In our original PF ruleset we have the following macros taken from the Calomel example:
[codeTcpState ="flags S/UAPRSF modulate state"
SshSTO ="(max 5, source-track rule, max-src-conn 5, max-src-nodes 5, max-src-conn-rate 5/30, overload <BLOCKTEMP> flush global)"[/code]
We have these default block rules:
We also have these specific rules further down:
However, some packets to and from nat'ed connections with [FONT=courier new]11.22.123.34[/FONT] are blocked by the default rules. Tcpdump shows this:
I cannot interpret the TCP flags and I have checked the timeouts but do not see anything unusual there:
Can someone explain to me what it is about 'flags S/UAPRSF modulate state' that causes the problem we are seeing? Or is this issue related to NAT'ing in some way? It seems that the stateful TCP connection initially established gets forgotten somehow. Then the following flags get treated as a new connect thus causing the failure. But I do not know why.
I do not understand TCP to any great extent so I cannot decipher what is causing PF to ignore the specific PF rules provided for the VAN. I can guess that there is something about the tcp flags or timeouts but I cannot interpret the traces. I tried simply removing the Calomel macros and just go with nothing on the rules, which should have given me '[FONT=courier new]keep state flags S/SA[/FONT]' . But that did not seem to work.
Switching to explicitly setting 'keep state flags S/SA' appears to eliminate the problem but I do not understand why this is necessary.
In our original PF ruleset we have the following macros taken from the Calomel example:
[codeTcpState ="flags S/UAPRSF modulate state"
SshSTO ="(max 5, source-track rule, max-src-conn 5, max-src-nodes 5, max-src-conn-rate 5/30, overload <BLOCKTEMP> flush global)"[/code]
We have these default block rules:
Code:
block return out log all
block drop in log all
We also have these specific rules further down:
Code:
pass in log quick \
from 11.22.123.34 \
to any $TcpState $SshSTO
pass out log quick \
from any \
to 11.22.123.34 $TcpState $SshSTO
Code:
00:00:00.035645 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.61121: Flags [.], ack 2086591712, win 160, length 0
00:00:00.008608 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.58929: Flags [.], ack 4167718526, win 160, length 0
00:00:00.048635 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.61121: Flags [.], ack 1, win 160, length 0
00:00:00.045995 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.58929: Flags [.], ack 1, win 160, length 0
00:00:00.176116 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.61121: Flags [.], ack 1, win 160, length 0
00:00:00.181632 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.58929: Flags [.], ack 1, win 160, length 0
00:00:00.037239 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.57395: Flags [.], ack 3896367661, win 160, length 0
00:00:00.160030 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.61121: Flags [.], ack 1, win 160, length 0
00:00:00.049054 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.58929: Flags [.], ack 1, win 160, length 0
00:00:00.130106 rule 1/0(match): block in on em1: 11.22.123.34.2148 > 75.232.5.45.57395: Flags [.], ack 1, win 160, length 0
I cannot interpret the TCP flags and I have checked the timeouts but do not see anything unusual there:
Code:
pfctl -s timeouts
tcp.first 120s
tcp.opening 30s
tcp.established 86400s
tcp.closing 900s
tcp.finwait 45s
tcp.closed 90s
tcp.tsdiff 30s
udp.first 60s
udp.single 30s
udp.multiple 60s
icmp.first 20s
icmp.error 10s
other.first 60s
other.single 30s
other.multiple 60s
frag 30s
interval 10s
adaptive.start 24000 states
adaptive.end 48000 states
src.track 0s
Can someone explain to me what it is about 'flags S/UAPRSF modulate state' that causes the problem we are seeing? Or is this issue related to NAT'ing in some way? It seems that the stateful TCP connection initially established gets forgotten somehow. Then the following flags get treated as a new connect thus causing the failure. But I do not know why.
Last edited by a moderator: