IPFW IPFW fwd rules have no effect

F

Falren

Well, hello again. Since nobody could fire out a reason for the disappearing handshakes, I figured that FreeBSD has enough firewalls to go around, so just try another one if the first doesn't work.

For background, I have a box running 11.0-RELEASE with the stock kernel. It has two interfaces, re0 (external) and em0 (internal), connected as a bridge0. The intent is to pass HTTP traffic from clients connected to em0 into a transparent Squid running on port 3128.

rc.conf
Code: 
firewall_enable="YES"
firewall_script="/usr/local/etc/ipfw.rules"
firewall_logging="YES"  
firewall_nat_enable="YES"


cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0 addm re0 SYNCDHCP"
ifconfig_em0="up"
ifconfig_re0="up"

gateway_enable="YES"

/usr/local/etc/ipfw.rules
Code: 
ruleadd="ipfw -q add"
ipfw -q -f flush
ipfw table all flush

# 192.168.0.157 is the system used for testing
$ruleadd 005 fwd 127.0.0.1,3128 log tcp from 192.168.0.157 to not me in via em0

$ruleadd 010 allow all from any to any

While /var/log/security does report packets to port 80 being forwarded to 127.0.0.1:3128, nothing shows up in Squid, the packets just pass through to their original destination. I've also tested it with nc -l 3128 to verify it's not a problem with Squid, again nothing.
I've also tried the looser rule suggested here, which seems to have worked for somebody with a similar setup, but no luck for me.

Do I have to try IPF next (never even used that), or does anybody have an idea why my fwd rules don't forward anything?
 
Please, may I see the output of # ifconfig

The most obvious difference between my setup and yours is, that my bridge members are not configured as inet interfaces by there own, but the bridge instead got assigned one IP address.
Code: 
cloned_interfaces="bridge0"
ifconfig_em0="up -tso"
ifconfig_em1="up -tso"
ifconfig_bridge0="inet $LANNET addm em0 addm em1 description LAN"
On my machine -- # ifconfig
Code: 
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: WAN
    options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
    ether 70:71:bc:63:3b:8c
    inet xxx.xy.yz.zz netmask 0xfffffc00 broadcast xxx.xy.yz.zz
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
    ether 00:11:22:33:44:55
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
    ether 00:11:22:33:44:56
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: LAN
    ether 00:11:22:33:44:57
    inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
    inet6 fd7d:40ea:3c93:b0a4::1 prefixlen 64
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: em1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
           ifmaxaddr 0 port 3 priority 128 path cost 55
    member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
           ifmaxaddr 0 port 2 priority 128 path cost 55
Because ipfw(8) does packet filtering also at the bridge layer, I added the following rules quite at the beginning of the rule set:
Code: 
...
/sbin/ipfw -q add 30 allow ip from any to any via em0
/sbin/ipfw -q add 31 allow ip from any to any via em1
...
In addition I set # systl net.inet.ip.fw.one_pass=0, mainly because mine is a NAT'ing firewall, and packets may & should pass more than once the ruleset. I never tested my bridge without NAT, however, I can imagine that this sysctl might be necessary for bridging as well.

Finally, like if_bridge(4) suggests, I have the following in my /boot/loader.conf:
Code: 
...
if_bridge_load="YES"
bridgestp_load="YES"
...
Although, I never tested, whether these modules are loaded automatically in the course of setting up the bridge by the rc script. You might want to check whether the bridge kernel modules are loaded using # kldstat:
Code: 
...
 3    2 0xffffffff821ed000 8f58     bridgestp.ko
 4    1 0xffffffff821f6000 11c38    if_bridge.ko
...
 
Thanks for the suggestions, although the one magic difference remains elusive. My setup actually is the same (minus the extra NIC), the bridge has an inet address and the member interfaces do not:
Code: 
# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
        ether 00:18:71:ea:dd:37
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether f4:6d:04:92:2f:24
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:fa:10:07:67:00
        inet 192.168.0.30 netmask 0xffffff00 broadcast 192.168.0.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 2000000


obsigna said:
Because ipfw(8) does packet filtering also at the bridge layer, I added the following rules quite at the beginning of the rule set:
Code: 
...
/sbin/ipfw -q add 30 allow ip from any to any via em0
/sbin/ipfw -q add 31 allow ip from any to any via em1
...
In addition I set # systl net.inet.ip.fw.one_pass=0, mainly because mine is a NAT'ing firewall, and packets may & should pass more than once the ruleset. I never tested my bridge without NAT, however, I can imagine that this sysctl might be necessary for bridging as well.
Click to expand...
Just to be clear, you have these two rules before the fwd rule?

Setting # systl net.inet.ip.fw.one_pass=0 does not seem to have an effect, and the bridge modules indeed seem to be loaded implicitly when creating a bridge,
kldstat confirms they are there.

Could it be that you have a routing entry which makes the whole thing work? My table is just the basics:
Code: 
#netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.0.1        UGS     bridge0
127.0.0.1          link#3             UH          lo0
192.168.0.0/24     link#4             U       bridge0
192.168.0.30       link#4             UHS         lo0
 
OK, so your fwd rule actually only sees packets on re0 and bridge0, the rest gets matched before.

*drumroll* No change when I try it, nothing shows up in Squid or nc even though /var/log/security claims stuff gets redirected:
Code: 
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50339 159.203.x.x:80 out via bridge0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50339 159.203.x.x:80 out via re0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50340 159.203.x.x:80 out via bridge0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50340 159.203.x.x:80 out via re0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50339 159.203.x.x:80 out via bridge0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50339 159.203.x.x:80 out via re0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50340 159.203.x.x:80 out via bridge0
Nov 29 22:23:21 xxx kernel: ipfw: 5 Forward to 127.0.0.1:3128 TCP 192.168.0.157:50340 159.203.x.x:80 out via re0
 
Falren said:
OK, so your fwd rule actually only sees packets on re0 and bridge0, the rest gets matched before.
...
Click to expand...

There must be a misunderstanding.

My early rules #30 and #31 take out all packets of my bridge members (em0 and em1) from further processing. In your case this would translate to:
Code: 
...
/sbin/ipfw -q add 30 allow ip from any to any via em0
/sbin/ipfw -q add 31 allow ip from any to any via re0
...
The fwd rule must not see packets from re0, it must see packets from bridge0 only.
 
Right, it should exclude both interfaces.

I did however stumble upon another possible angle: HTTP requests tunneled through an SSH connection to the box do get redirected. So maybe aliasing requests to the address of the bridge before the forwarding rule is the solution...not ideal because the loss of transparency, but it would work for the intended use case.

I'll report back once I got the chance to test this.
 
You must log in or register to reply here.
Back
Top