Hello,
I've got a PC with 7 NICs (rl0, vr0, re0-re4 - see please pf.conf for details) which I use as a router and firewall to connect 2 LANs a DMZ via 3 gateways to the internet. I'm using FreeBSD 9.1 and pf (current version). So far, so good - everything works fine. I use 3 NICs for 3 different gateways (2 with fixed IPs, one dynamic) to the internet. From 2 LANs I could balance and queue traffic between those 3 gateways (gw0-gw2), and I'm also able to rdr - BUT, and that's the point, not on all NICs, just on one. The gw0 works, the gw1 not (gw2 is not used for rdr, because of the dynamic IP). There is definitely no reaction on that interface / NIC if you try to access it from outside (tried everything, nmap als means the host is probably down). But, traffic to the outside, over that interface works fine via NAT.
My answer to that is: "Bug in pf, bug in FreeBSD ..." or do you have a different opinion, some other options, etc? So please help me in that case, thank you
ps.
and
I've got a PC with 7 NICs (rl0, vr0, re0-re4 - see please pf.conf for details) which I use as a router and firewall to connect 2 LANs a DMZ via 3 gateways to the internet. I'm using FreeBSD 9.1 and pf (current version). So far, so good - everything works fine. I use 3 NICs for 3 different gateways (2 with fixed IPs, one dynamic) to the internet. From 2 LANs I could balance and queue traffic between those 3 gateways (gw0-gw2), and I'm also able to rdr - BUT, and that's the point, not on all NICs, just on one. The gw0 works, the gw1 not (gw2 is not used for rdr, because of the dynamic IP). There is definitely no reaction on that interface / NIC if you try to access it from outside (tried everything, nmap als means the host is probably down). But, traffic to the outside, over that interface works fine via NAT.
My answer to that is: "Bug in pf, bug in FreeBSD ..." or do you have a different opinion, some other options, etc? So please help me in that case, thank you
ps.
- I did all my homework
- everything is set and activated like it should (rc.conf, sysctl.conf, rc.local...), all NICs are up and working fine (with) and without pf enabled. Else, rdr wouldn't work with gw0 too ;-) ...
- I checked everything outside the wall (that means, the router from my provider, which is connected to gw1 works fine. If I connect an other device, like f.e. a Zywall 2, it will work like it should (you can access server B)
- DMZ is not in use - Server A and B are located in the LAN because of development purpose
- The problem seems to be here:
- but this rule does not really differ from the gw0 ruleCode:
rdr pass on $gw1_if proto tcp from any to any port { 80, 22 } -> $srvB - It makes no difference if you delete all filter rules and use just
- there will be still no reaction on gw1 if you rdrCode:
pass in all pass out all
Code:
gw0_if="re4"
gw1_if="rl0"
gw2_if="vr0"
lan1_if="re3"
lan2_if="re2"
dmz_if="re1"
gw0_net="10.0.0.0/24"
gw1_net="10.0.1.0/24"
gw2_net="10.0.2.0/24"
lan1_net="192.168.0.0/24"
lan2_net="192.168.1.0/24"
dmz_net="192.168.2.0/24"
gw0="10.0.0.138"
gw1="10.0.1.138"
gw2="10.0.2.138"
lan1="192.168.0.1"
lan2="192.168.1.1"
dmz="192.168.2.1"
srvA="192.168.0.54"
srvB="192.168.0.50"
table <lan1_trusted> const { 192.168.0. ...}
table <lan1_prio> const { 192.168.0. ...}
table <gw0_trusted> const { 192.168.0. ...}
table <gw0_mandatory> const { 192.168.0. ...}
table <gw1_trusted> const { 192.168.0. ...}
table <gw0_routed> const { a.b.c.d ...}
table <gw1_routed> const { a.b.c.d ...}
table <gw_admin> const { 192.168.0. ... }
table <lan1_master> const { 192.168.0. ... }
table <ssh_trusted> const { 192.168.0. ... a.b.c.d ...}
table <peter> const { 80.109.197.142 }
t_p = "{tcp, udp, icmp}"
d_p = "{tcp, udp}"
# 3) Queue
# --------------- GW0 (nach Internet auf 8/1 DIC)
altq on $gw0_if cbq bandwidth 960Kb queue { gw0_def, gw0_lan1, gw0_dmz}
queue gw0_def bandwidth 70% cbq(default)
queue gw0_lan1 bandwidth 10% { gw0_lan1_pri, gw0_lan1_oth }
queue gw0_lan1_pri bandwidth 70% priority 4 cbq(red borrow)
queue gw0_lan1_oth bandwidth 30% priority 1 cbq(red borrow)
queue gw0_dmz bandwidth 20% priority 6 cbq(red borrow)
# --------------- GW1 (nach Internet auf 16/1 DIC)
altq on $gw1_if cbq bandwidth 4Mb queue { gw1_def, gw1_lan1, gw1_lan2, gw1_dmz}
queue gw1_def bandwidth 50% cbq(default)
queue gw1_lan1 bandwidth 20% { gw1_lan1_pri, gw1_lan1_oth }
queue gw1_lan1_pri bandwidth 70% priority 6 cbq(red borrow)
queue gw1_lan1_oth bandwidth 30% priority 3 cbq(red borrow)
queue gw1_lan2 bandwidth 10% priority 1 cbq(red borrow)
queue gw1_dmz bandwidth 20% priority 4 cbq(red borrow)
# --------------- GW2 (nach Internet auf 16/1 Tele2)
altq on $gw2_if cbq bandwidth 960Kb queue { gw2_def, gw2_lan1, gw2_lan2}
queue gw2_def bandwidth 70% cbq(default)
queue gw2_lan1 bandwidth 15% { gw2_lan1_pri, gw2_lan1_oth }
queue gw2_lan1_pri bandwidth 60% priority 3 cbq(red borrow)
queue gw2_lan1_oth bandwidth 40% priority 1 cbq(red borrow)
queue gw2_lan2 bandwidth 15% priority 2 cbq(red borrow)
# 4) NAT
# --------------- lan1 -> Internet
nat on $gw0_if from $lan1_if:network to { any, !$lan1_net, !$dmz_net, !$gw1_net, !$gw2_net } -> ($gw0_if:0)
nat on $gw1_if from $lan1_if:network to { any, !$lan1_net, !$dmz_net, !$gw0_net, !$gw2_net } -> ($gw1_if:0)
nat on $gw2_if from $lan1_if:network to { any, !$lan1_net, !$dmz_net, !$gw1_net, !$gw0_net } -> ($gw2_if:0)
# --------------- lan2 -> Internet
nat on $gw1_if from $lan2_if:network to { any, !$dmz_net } -> ($gw1_if:0)
nat on $gw2_if from $lan2_if:network to { any, !$dmz_net } -> ($gw2_if:0)
# --------------- DMZ -> Internet
nat on $gw0_if from $dmz_if:network to any -> ($gw0_if:0)
nat on $gw1_if from $dmz_if:network to any -> ($gw1_if:0)
# 5) RDR
# --------------- srvA - production
rdr pass on $gw0_if proto tcp from any to any port { 25, 465, 143, 993, 587, 80, 443, 22 } -> $srvA
# --------------- srvB - development
rdr pass on $gw1_if proto tcp from any to any port { 80, 22 } -> $srvB
# 6)
pass in quick from <lan1_trusted> to $lan1
antispoof quick for { lo $gw0_if $gw1_if $gw2_if $lan2_if}
# 7) for Loadbalan1cing & Queuing
pass out quick on $gw0_if from <gw0_mandatory> keep state queue(gw0_def, gw0_dmz)
# ---------------
pass out quick on $gw2_if from <lan1_prio> keep state queue(gw2_lan1_pri, gw2_def)
pass out quick on $gw2_if from $lan1_net keep state queue(gw2_lan1_oth, gw2_def)
pass out quick on $gw2_if from $lan2_net keep state queue(gw2_lan2, gw2_def)
pass out quick on $gw1_if from <lan1_prio> keep state queue(gw1_def, gw1_lan1_pri)
pass out quick on $gw1_if from $lan1_net keep state queue(gw1_def, gw1_lan1_oth)
pass out quick on $gw1_if from $dmz_net keep state queue(gw1_def, gw1_dmz)
pass out quick on $gw0_if from <lan1_prio> keep state queue(gw0_def, gw0_lan1_pri)
pass out quick on $gw0_if from $lan1_net keep state queue(gw0_def, gw0_lan1_oth)
pass out quick on $gw0_if from $dmz_net keep state queue(gw0_def, gw0_dmz)
# ---------------
pass out on $gw0_if route-to ($gw1_if $gw1) from $gw1_if
pass out on $gw1_if route-to ($gw0_if $gw0) from $gw0_if
pass out on $gw1_if route-to ($gw2_if $gw2) from $gw2_if
pass out on $gw2_if route-to ($gw1_if $gw1) from $gw1_if
# 8) special traffic
# --------------- lan1 --> DMZ and lan2
pass in quick on $lan1_if from $lan1_net to $dmz_if keep state
pass in quick on $lan1_if from $lan1_net to $lan2_if keep state
# --------------- lan1 --> special
pass in quick on $lan1_if from <gw0_trusted> to <gw0_routed> keep state rtable 0
pass in quick on $lan1_if from <gw_admin> to $gw0_net keep state rtable 0
pass in quick on $lan1_if from <gw1_trusted> to <gw1_routed> keep state rtable 1
pass in quick on $lan1_if from <gw_admin> to $gw1_net keep state rtable 1
pass in quick on $lan1_if from <gw_admin> to $gw2_net keep state rtable 2
# --------------- lan1 --> Internet
pass in quick on $lan1_if route-to { ($gw1_if $gw1), ($gw2_if $gw2), ($gw0_if $gw0) } round-robin from <lan1_prio> to any keep state
pass in quick on $lan1_if route-to { ($gw1_if $gw1), ($gw2_if $gw2) } round-robin from $lan1_net to any keep state
# --------------- lan2 --> Internet
block in quick on $lan2_if from any to $gw0_if
pass in quick on $lan2_if route-to { ($gw2_if $gw2), ($gw1_if $gw1) } round-robin from $lan2_net to any keep state
# 9) IN
# --------------- GW0 -> srvA
pass in quick on $gw0_if proto TCP from any to $srvA port { 25, 465, 143, 993, 587, 80, 22 } keep state rtable 0
# --------------- GW1 (vi lan1) -> srvB
pass in quick on $gw1_if proto TCP from any to $srvB port { 22, 80 } keep state rtable 1
# 10) BLOCK
block in on $gw0_if all
block in on $gw1_if all
block in on $gw2_if all
block in on $lan2_if all# setfib 0 netstat -nr -f inet produces this
Code:
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 10.0.0.138 UGS 0 167 re4
10.0.0.0/24 link#5 U 0 5334 re4
10.0.0.1 link#5 UHS 0 2 lo0
10.0.1.0/24 link#7 U 0 1019 rl0
10.0.1.1 link#7 UHS 0 0 lo0
10.0.2.0/24 link#6 U 0 14 vr0
10.0.2.1 link#6 UHS 0 0 lo0
127.0.0.1 link#10 UH 0 676 lo0
192.168.0.0/24 link#4 U 0 3719388 re3
192.168.0.1 link#4 UHS 0 2 lo0
192.168.1.0/24 link#3 U 0 4066489 re2
192.168.1.1 link#3 UHS 0 0 lo0
192.168.2.0/24 link#2 U 0 0 re1
192.168.2.1 link#2 UHS 0 0 lo0 setfib 1 and 2 the same with different gates/default routes (10.0.1.138 and 10.0.2.138)