This issue has had me stumped for month or two now. I've been running a home-built router in front of my cable modem that is running 11.2-stable. -stable because I hit an issue with console initialisation that IIRC had been patched in -stable but not -release at the point when I last rebuilt the box.
This firewall box has been running a simple pf firewall configuration for a long time with no noticable hiccups. The pf config is as follows:
I then tried to duplicate the above pf configuration and translated it to ipfw as the first step:
On the face of it, the ipfw configuration works. It looks like it nats correctly and I can see the expected firewall rules being dynamically generated as connections are being opened. However the firewall isn't reliable at that point and seems to be randomly deciding to establish connections (mostly http/s) or not. A typical example is that I'm browsing a forum and switching to the next page will require multiple reloads until the connection is finally established. I haven't wireshark'd the traffic between the router and the cable modem yet but to me this looks very much like the responses from outside servers are either not being properly nat'd back or are filtered out by firewall rules that shouldn't be filtering them out. Commenting out the line
improves matters somewhat, but doesn't fix the issue. This leads me to believe the issue is the in-kernel NAT, not the firewall rules themselves.
Searching the forum brought up a couple of posts discussing that TSO doesn't play nicely with ipfw's kernel NAT. ifconfig's output shows that TSO is not enabled on any of the interfaces on the system (em0 being the WAN facing interface):
I've got the distinct feeling that I'm missing something blindingly obvious here, but I don't seem to be able to figure out exactly where the misconfiguration resides.
This firewall box has been running a simple pf firewall configuration for a long time with no noticable hiccups. The pf config is as follows:
Code:
ext_if = "em0"
int_if = "{ em1, em2, em2 }"
localnet = "{ em1:network, em2:network, em3:network }"
set skip on lo0
scrub in all
nat on $ext_if from $localnet to any -> ($ext_if)
block in
pass out
pass quick on $int_if no state
antispoof quick for $int_if
pass from { lo, em1:network, em2:network, em3:network } to any keep stateI then tried to duplicate the above pf configuration and translated it to ipfw as the first step:
Code:
lan_if="em1, em2, em3"
wan_if="em0"
#netbios="137,138,139"
setup_loopback() {
############
# Only in rare cases do you want to change these rules
#
${fwcmd} add pass all from any to any via lo0
${fwcmd} add deny all from any to 127.0.0.0/8
${fwcmd} add deny ip from 127.0.0.0/8 to any
if [ $ipv6_available -eq 0 ]; then
${fwcmd} add deny all from any to ::1
${fwcmd} add deny all from ::1 to any
fi
}
${fwcmd} -f flush
${fwcmd} nat 1 config if ${wan_if} same_ports reset
${fwcmd} add reass all from any to any in
# Throw in default configurations for loopback
setup_loopback
${fwcmd} add allow ip from any to any via em1
${fwcmd} add allow ip from any to any via em2
${fwcmd} add allow ip from any to any via em3
${fwcmd} add deny ip from any to any not verrevpath in
${fwcmd} add nat 1 ip from any to any via ${wan_if} in
${fwcmd} add check-state
${fwcmd} add deny log tcp from any to any established via ${wan_if}
${fwcmd} add skipto 32000 tcp from any to any via ${wan_if} out setup keep-state
${fwcmd} add skipto 32000 udp from any to any via ${wan_if} out keep-state
${fwcmd} add pass icmp from any to any
${fwcmd} add deny tcp from any to any via ${wan_if}
${fwcmd} add deny udp from any to any via ${wan_if}
${fwcmd} add 32000 nat 1 ip from any to any via ${wan_if} outOn the face of it, the ipfw configuration works. It looks like it nats correctly and I can see the expected firewall rules being dynamically generated as connections are being opened. However the firewall isn't reliable at that point and seems to be randomly deciding to establish connections (mostly http/s) or not. A typical example is that I'm browsing a forum and switching to the next page will require multiple reloads until the connection is finally established. I haven't wireshark'd the traffic between the router and the cable modem yet but to me this looks very much like the responses from outside servers are either not being properly nat'd back or are filtered out by firewall rules that shouldn't be filtering them out. Commenting out the line
Code:
${fwcmd} add deny log tcp from any to any established via ${wan_if}Searching the forum brought up a couple of posts discussing that TSO doesn't play nicely with ipfw's kernel NAT. ifconfig's output shows that TSO is not enabled on any of the interfaces on the system (em0 being the WAN facing interface):
Code:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrierI've got the distinct feeling that I'm missing something blindingly obvious here, but I don't seem to be able to figure out exactly where the misconfiguration resides.
