ipfw ruleset/nat is occasionally losing TCP responses

[pkc]

Others on the network have noticed that occasionally, web pages will simply fail to load. I checked out one instance of this on tcpdump: the request makes it out, and a response comes back on the external interface but does not make it back to the original host. If I recall correctly the responses contained FINs as well. Is this a problem with my ruleset, or is there maybe some buffer/queue that is being filled up somewhere? I should note that the responses are not appearing in the deny log (ipfw0, which I was monitoring with tcpdump). Any other suggestions on the ruleset are appreciated as well. Thanks!

EDIT
It seems that FINs from the host are not making it through the firewall, because of the 'deny tcp ... established' rule after check-state. This could be causing subsequent connections to fail in the way that I observed above. The "check-state..deny tcp established" idiom is something I found online so I will disable that for now.

Here's my ruleset

ipfw -q -f flush
add="ipfw add"

intif="re0"
intnet="192.168.1.0/16"
inthost="192.168.1.254"

extif="re1"
exthost="1.2.3.4"

# default rules
$add    10      allow ip from any to any via lo0
$add    20      deny ip from any to 127.0.0.1/8
$add    30      deny ip from 127.0.0.1/8 to any
$add    40      deny ip from any to ::1
$add    50      deny ip from ::1 to any
$add    60      allow ipv6-icmp from :: to ff02::/16
$add    70      allow ipv6-icmp from fe80::/10 to fe80::/10
$add    80      allow ipv6-icmp from fe80::/10 to ff02::/16
$add    90      allow ipv6-icmp from any to any ip6 icmp6types 1
$add    100     allow ipv6-icmp from any to any ip6 icmp6types 2,135,136


# Local network
$add    120     allow all from $inthost to any via $intif
$add    121     allow all from $intnet to $inthost via $intif


$add divert natd all from any to any via $extif

# Outgoing UDP & ICMP
$add allow { icmp or udp } from { $intnet or $exthost } to any 

# ICMP responses to $intnet, 
$add allow icmp from any to { $intnet or $exthost }

# UDP responses
$add allow udp from any to { $intnet or $exthost } src-port 53
$add allow udp from any to { $intnet or $exthost } src-port 123

# Outgoing TCP
$add    790     check-state
$add    800     deny tcp from any to any established
$add allow tcp from { $intnet or $exthost } to any setup keep-state

$add    10000   deny log all from any to any

 

[obsigna]

For building a NAT'ting stateful firewall with ipfw(8), you need to embrace the check-state by two NAT rules, see:
http://www.freebsd.org/doc/en_US.ISO885 ... twork-natd

...
$add 100 divert natd ip from any to any in via $extif
$add 101 check-state
...
$add 700 skipto 10000 tcp from any to any via $extif out setup keep-state
$add 701 skipto 10000 udp from any to any via $extif out keep-state
...
$add 9998 deny tcp from any to any via $extif
$add 9999 deny udp from any to any via $extif
$add 10000 divert natd ip from any to any out via $extif
...
$add 65534 allow ip from any to any

The stateful rules for outgoing traffic must go after the check-state rule and before the outgoing NAT rule, and these outgoing rules should use skipto 10000 (rule number of the outgoing NAT rule) instead of allow.

 

[pkc]

For building a NAT'ting stateful firewall with ipfw(8), you need to embrace the check-state by two NAT rules, see:
http://www.freebsd.org/doc/en_US.ISO885 ... twork-natd

...
$add 100 divert natd ip from any to any in via $extif
$add 101 check-state
...
$add 700 skipto 10000 tcp from any to any via $extif out setup keep-state
$add 701 skipto 10000 udp from any to any via $extif out keep-state
...
$add 9998 deny tcp from any to any via $extif
$add 9999 deny udp from any to any via $extif
$add 10000 divert natd ip from any to any out via $extif
...
$add 65534 allow ip from any to any

The stateful rules for outgoing traffic must go after the check-state rule and before the outgoing NAT rule, and these outgoing rules should use skipto 10000 (rule number of the outgoing NAT rule) instead of allow.

Thanks for the reply! I managed to get it working using your advice. However, for some reason it seems like TCP sessions will be terminated after some inactivity. (eg ssh). I can investigate this.

 

[obsigna]

...
Thanks for the reply! I managed to get it working using your advice. However, for some reason it seems like TCP sessions will be terminated after some inactivity. (eg ssh). I can investigate this.

This happens because the dynamic rules which are created by the keep-state directive, got a limited lifetime, when there is no traffic. For this reason many processes send frequently keep-alive dummy packets. In the case of ssh the keep-alive shall come from the ssh-client in order to trigger the respective dynamic firewall rule. If you use the ssh client from OpenSSH on a *BSD or on a Mac OS X system, then sending keep-alive packets is active by default. When using PuTTY on Windows, you may want to put a value like 120 into the field Seconds between keepalives in the Connection options pane.