Hi.
I am using PF on a commercial server which is used for both irc and web services.
The main purpose of the PF rules is to stop people on irc accounts from using web ip addresses and to rate limit connections to sshd and web services.
The problem I am having is after a random period of time timeouts will start occuring on new connections, these timouts do not occur on rate limited connections, they are occuring on irc services. They stop occuring when PF is turned off. In addition nothing appears in the PF logs when they occur.
I have only been using PF for around 6 months, there is a severe lack of documentation and discussion on it when compared to ipfw so its very much a learning curve for me, I will provide my ruleset below with usernames and ip's censored out. My ip is 3rd in the trusted ip list and still gets random timeouts when PF is up, suggesting its either the scrub or the stateful part of the firewall having problems.
Some notes about the following rules.
(a) keep-state on the user rules was originally flags S/SA
(b) allow-opts wasnt originally in place
(c) scrub was originally set to 'scrub in on $ext_if fragment reassemble'
(d) I am noticing no problems on the rules above the user rules, eg. sshd is always accessible, www is always accessible and so on.
(e) -edit- sorry to add this is a FreeBSD 6.3 machine so is not the newest PF version.
The timeouts occur on the rules at the end that restrict what ips can be used for outbound connections, there is not an outright blocking of traffic instead connections will timeout and then work again, then timeout then work again and so on.
The S/SA flags were removed of the user rules as I dont want those connections to drop when I flush the states, however they still drop.
Finally another strange occurance is that if I do pfctl -s info this ' searches 104376 0.1/s' constantly increases the rate, so this line I pasted is right after I did a flush and is 0.1/sec if I leave PF running without flushing the rate of searches keeps going up and up and after a few days is about 200/sec.
So summary.
1 - how to stop stateful connections dropping when flush states, seems according to docs this can be done by allowing any packet to create a new state but when I removed flags S/SA this didnt do this.
2 - how to stop random new timeouts when trying to establish new connections, seems to only be affecting the last rules which restrict users.
3 - why is the state search rate counter getting higher with uptime.
4 - how to fully disable scrubbing, to see if this is why I get random timeouts.
5 - any obvious problems with the rules please point out thanks.
ext_if="bge0"
int_if="lo0"
trusted = "{ 208.x.x.x, 85.x.x.x, 87.x.x.x }"
someuser1 = "{ 85.x.x.x, 85.x.x.x, 85.x.x.x, 85.x.x.x }"
someuser2 = "{ 85.x.x.x, 85.x.x.x, 85.x.x.x }"
someuser3 = "{ 85.x.x.x }"
someuser4 = "{ 85.x.x.x }"
someuser5 = "{ 85.x.x.x }"
someuser6 = "{ 85.x.x.x }"
someuser7 = "{ 85.x.x.x }"
main="85.x.x.x"
default="{ <all server ips here except first one> }"
icmp_types="{ echoreq, unreach }"
table <badhosts> persist
table <allowed> { <all server ips here including first one> }
set loginterface $ext_if
set optimization normal
set block-policy drop
set state-policy if-bound
set require-order yes
set fingerprints "/etc/pf.os"
set debug misc
set skip on lo0
scrub on $ext_if reassemble tcp
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
rdr pass on $int_if proto tcp from any to any port ftp -> 127.0.0.1 \
port 8021
pass in quick on $ext_if proto { tcp, udp, icmp } from $trusted to any keep state
block drop in log all
block return out log all
block quick on $ext_if from <badhosts> to any
antispoof for $ext_if inet
anchor "ftp-proxy/*"
# Allow for server -> outside connections
pass out on $ext_if proto { udp, gre, icmp } from any to any keep state
pass out on $ext_if proto tcp from $ext_if to any flags S/SA keep state
pass in on $ext_if inet proto tcp from any to $ext_if \
port 10100:10200 keep state
pass in inet proto icmp all icmp-type $icmp_types keep state
# Port opening
pass in quick on $ext_if proto udp from 83.x.x.x to $main port 161
pass in quick on $ext_if proto udp from any to $default port 53
pass in on $ext_if proto tcp from any to $default flags S/SA keep state
pass in quick on $ext_if proto tcp from any to $main port { 42, 25, 2222, 3306, 21, 587, 143, 465, 110, 993, 995, 53, 20000 } flags S/SA keep state (max-src-conn-rate 20/60, overload <badhosts> flush global)
pass in quick on $ext_if proto tcp from any to $ext_if port { 80, 443 } flags S/SA synproxy state (max-src-conn-rate 100/10, overload <badhosts> flush global)
#user name restrictions
block out log on $ext_if proto tcp from any to any user someuser
pass out quick on $ext_if proto tcp from $someuser to any allow-opts user someuser
block out log on $ext_if proto tcp from any to any user someuser2
pass out on $ext_if proto tcp from $someuser2 to any user someuser2 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser3
pass out on $ext_if proto tcp from $someuser3 to any user someuser3 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser4
pass out on $ext_if proto tcp from $someuser4 to any user someuser4 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser5
pass out on $ext_if proto tcp from $someuser5 to any user someuser5 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser6
pass out on $ext_if proto tcp from $someuser6 to any user someuser6 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser7
pass out on $ext_if proto tcp from $someuser7 to any user someuser7 allow-opts keep state
I am using PF on a commercial server which is used for both irc and web services.
The main purpose of the PF rules is to stop people on irc accounts from using web ip addresses and to rate limit connections to sshd and web services.
The problem I am having is after a random period of time timeouts will start occuring on new connections, these timouts do not occur on rate limited connections, they are occuring on irc services. They stop occuring when PF is turned off. In addition nothing appears in the PF logs when they occur.
I have only been using PF for around 6 months, there is a severe lack of documentation and discussion on it when compared to ipfw so its very much a learning curve for me, I will provide my ruleset below with usernames and ip's censored out. My ip is 3rd in the trusted ip list and still gets random timeouts when PF is up, suggesting its either the scrub or the stateful part of the firewall having problems.
Some notes about the following rules.
(a) keep-state on the user rules was originally flags S/SA
(b) allow-opts wasnt originally in place
(c) scrub was originally set to 'scrub in on $ext_if fragment reassemble'
(d) I am noticing no problems on the rules above the user rules, eg. sshd is always accessible, www is always accessible and so on.
(e) -edit- sorry to add this is a FreeBSD 6.3 machine so is not the newest PF version.
The timeouts occur on the rules at the end that restrict what ips can be used for outbound connections, there is not an outright blocking of traffic instead connections will timeout and then work again, then timeout then work again and so on.
The S/SA flags were removed of the user rules as I dont want those connections to drop when I flush the states, however they still drop.
Finally another strange occurance is that if I do pfctl -s info this ' searches 104376 0.1/s' constantly increases the rate, so this line I pasted is right after I did a flush and is 0.1/sec if I leave PF running without flushing the rate of searches keeps going up and up and after a few days is about 200/sec.
So summary.
1 - how to stop stateful connections dropping when flush states, seems according to docs this can be done by allowing any packet to create a new state but when I removed flags S/SA this didnt do this.
2 - how to stop random new timeouts when trying to establish new connections, seems to only be affecting the last rules which restrict users.
3 - why is the state search rate counter getting higher with uptime.
4 - how to fully disable scrubbing, to see if this is why I get random timeouts.
5 - any obvious problems with the rules please point out thanks.
ext_if="bge0"
int_if="lo0"
trusted = "{ 208.x.x.x, 85.x.x.x, 87.x.x.x }"
someuser1 = "{ 85.x.x.x, 85.x.x.x, 85.x.x.x, 85.x.x.x }"
someuser2 = "{ 85.x.x.x, 85.x.x.x, 85.x.x.x }"
someuser3 = "{ 85.x.x.x }"
someuser4 = "{ 85.x.x.x }"
someuser5 = "{ 85.x.x.x }"
someuser6 = "{ 85.x.x.x }"
someuser7 = "{ 85.x.x.x }"
main="85.x.x.x"
default="{ <all server ips here except first one> }"
icmp_types="{ echoreq, unreach }"
table <badhosts> persist
table <allowed> { <all server ips here including first one> }
set loginterface $ext_if
set optimization normal
set block-policy drop
set state-policy if-bound
set require-order yes
set fingerprints "/etc/pf.os"
set debug misc
set skip on lo0
scrub on $ext_if reassemble tcp
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
rdr pass on $int_if proto tcp from any to any port ftp -> 127.0.0.1 \
port 8021
pass in quick on $ext_if proto { tcp, udp, icmp } from $trusted to any keep state
block drop in log all
block return out log all
block quick on $ext_if from <badhosts> to any
antispoof for $ext_if inet
anchor "ftp-proxy/*"
# Allow for server -> outside connections
pass out on $ext_if proto { udp, gre, icmp } from any to any keep state
pass out on $ext_if proto tcp from $ext_if to any flags S/SA keep state
pass in on $ext_if inet proto tcp from any to $ext_if \
port 10100:10200 keep state
pass in inet proto icmp all icmp-type $icmp_types keep state
# Port opening
pass in quick on $ext_if proto udp from 83.x.x.x to $main port 161
pass in quick on $ext_if proto udp from any to $default port 53
pass in on $ext_if proto tcp from any to $default flags S/SA keep state
pass in quick on $ext_if proto tcp from any to $main port { 42, 25, 2222, 3306, 21, 587, 143, 465, 110, 993, 995, 53, 20000 } flags S/SA keep state (max-src-conn-rate 20/60, overload <badhosts> flush global)
pass in quick on $ext_if proto tcp from any to $ext_if port { 80, 443 } flags S/SA synproxy state (max-src-conn-rate 100/10, overload <badhosts> flush global)
#user name restrictions
block out log on $ext_if proto tcp from any to any user someuser
pass out quick on $ext_if proto tcp from $someuser to any allow-opts user someuser
block out log on $ext_if proto tcp from any to any user someuser2
pass out on $ext_if proto tcp from $someuser2 to any user someuser2 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser3
pass out on $ext_if proto tcp from $someuser3 to any user someuser3 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser4
pass out on $ext_if proto tcp from $someuser4 to any user someuser4 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser5
pass out on $ext_if proto tcp from $someuser5 to any user someuser5 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser6
pass out on $ext_if proto tcp from $someuser6 to any user someuser6 allow-opts keep state
block out log on $ext_if proto tcp from any to any user someuser7
pass out on $ext_if proto tcp from $someuser7 to any user someuser7 allow-opts keep state