Port redirection on 2 WAN setup

J

jailed

Hello,

I have a server acting as a router. I use FreeBSD 9.1 (i386) and PF on it. It has three network interfaces. Two of them are for WAN and remaining one is for LAN. I have no problem with NAT. It's working without problem. The problem is, I cannot accept incoming TCP connections from the second WAN.

Here's the necessary parts of my pf.conf
Code: 
table <network_1> persist { 192.168.0.1, 192.168.0.2, 192.168.0.50, 192.168.0.51, 192.168.0.52, 192.168.0.53 }
table <network_1_area> persist { 192.168.0.0/24 }

table <network_2> persist { 192.168.0.3, 192.168.0.4, 192.168.0.5, 192.168.0.20 }

wan_2_tcp_incoming = "{ 80 }"

icmp_types = "{ echoreq, unreach }"

set block-policy return
set loginterface $wan_1
set state-policy if-bound
set skip on lo0

# NAT
nat on $wan_1 from <network_1> to any -> $ip_1
nat on $wan_2 from <network_2> to any -> $ip_2

rdr pass on $wan_1 proto tcp from any to $ip_1 port 80 -> 192.168.0.2 port 80
rdr pass on $wan_1 proto tcp from any to $ip_1 port 8000 -> 192.168.0.2 port 8000
rdr pass on $wan_1 proto tcp from any to $ip_1 port 443 -> 192.168.0.2 port 443
rdr pass on $wan_1 proto tcp from any to $ip_1 port 2222 -> 192.168.0.1 port 22

rdr pass log on $wan_2 proto tcp from any to $ip_2 port 80 -> 192.168.0.3 port 80 # Not working

pass in on $lan all
block in log on ste0 all
block in log on ste1 all

pass in on $wan_1 inet proto tcp from any to $ip_1 port $wan_1_tcp_incoming
pass in on $wan_1 inet proto udp from any to $ip_1 port $wan_1_udp

# pass in log on $wan_2 reply-to ( $wan_2 $gw_2 ) proto tcp from any to $ip_2 port $wan_2_tcp_incoming

pass in inet proto udp from any port 53 to any

pass inet proto icmp all icmp-type $icmp_types

pass in on $lan route-to ( $wan_1 $gw_1 ) from <network_1> to !<network_1_area> keep state
pass in on $lan route-to ( $wan_2 $gw_2 ) from <network_2> to !<network_1_area> keep state

pass in on $wan_1 route-to ( $wan_2 $gw_2 ) from <network_2> keep state
pass in on $wan_2 route-to ( $wan_1 $gw_1 ) from <network_1> keep state

pass out on $wan_1 route-to ( $wan_2 $gw_2 ) from $wan_2 to any keep state
pass out on $wan_2 route-to ( $wan_1 $gw_1 ) from $wan_1 to any keep state

pass out all

# END

The redirection on the port 80 of $wan_2 is not working. The packet reaches to 192.168.0.3 but the reply doesn't reach to the source address. When I disable port forwarding and directly bind httpd to $ip_2 on the router, it still does not reply to the source.

I will be very happy if anyone can help me solve this problem.

Thanks.
 
jailed said:
The packet reaches 192.168.0.3 but the reply doesn't reach to the source address. When I disable port forwarding and directly binding httpd to $ip_2 on the router, it still does not reply to the source.
Click to expand...
This is most likely a routing problem. Packets coming in on WAN2 might have their replies sent out on WAN1 depending on where your default gateway is set to.
 
kpa said:
Isn't reply-to tailor made for this situation?
Click to expand...

It should, yes. Although I never quite understood how to use it properly. I tend to prevent situations like this from happening ;)
 
This appears to be a routing issue. Try running a tcpdump to make sure.

You can create multiple route paths using setfib(1)() but you also need to compile your kernel with:

Code: 
options         ROUTETABLES=4

Then you add each gateway: # setfib 1 route add -net default 10.0.0.1. I had done this in the past but I can't recall exactly how...
 
Thank you for your replies. Previously, I tried the rtable method, but it didn't work for me. I couldn't understand how it works. There's no enough documentation on the Internet. I have also tried with OpenBSD, but the results were the same.

I created all the routing rules correctly with the route-to command. Different IP's on the same LAN can use different ISPs with my configuration. Only this port redirection does not work. I think it's easy to solve with PF's route-to and reply-to commands. I don't understand how the reply-to command really works, so that I couldn't solve the last piece of the job.

I've seen some examples with tag and tagged rules. This can help too, but there's not enough documentation with this topics too.

I'm also considering buying a hardware router/switch. But I'm not sure which router can help me for this setup. I have two Internet connections for now. But I can increase the amount of it in the future. So, using a server with PF is a cheaper solution for now.
 
The reply-to should be applied to the filter rules that accept the redirected traffic. So something like what you have already done but have commented out:

Code: 
rdr log on $wan_2 proto tcp from any to ($wan_2) port 80 -> 192.168.0.3 port 80 
...
pass in log on $wan_2 reply-to ( $wan_2 $gw_2 ) proto tcp from any to 192.168.0.3 port 80

I took out the pass from the rdr because it creates another filter rule that competes with the rule below it.
 
Yes, reply-to is certainly the answer. I just made my router/firewall dual-homed yesterday, and the second WAN interface (i.e. not the default gateway) needed reply-to in its pass in rules.

E.g.

Code: 
pass in quick on $second_wan_if reply-to ( $second_wan_if $second_wan_ip ) inet proto udp from any to $second_wan_if port 53 keep state
 
Thank you for your replies. But it's not working.

Here's the latest version of my pf.conf:
Code: 
table <network_1> persist { 192.168.0.1, 192.168.0.2, 192.168.0.50, 192.168.0.51, 192.168.0.52, 192.168.0.53 }
table <network_1_area> persist { 192.168.0.0/24 }

table <network_2> persist { 192.168.0.3, 192.168.0.4, 192.168.0.5, 192.168.0.20 }

wan_2_tcp_incoming = "{ 80 }"

icmp_types = "{ echoreq, unreach }"

set block-policy return
set loginterface $wan_1
set state-policy if-bound
set skip on lo0

# NAT
nat on $wan_1 from <network_1> to any -> $ip_1
nat on $wan_2 from <network_2> to any -> $ip_2

# RDR
rdr pass on $wan_1 proto tcp from any to $ip_1 port 80 -> 192.168.0.2 port 80
rdr pass on $wan_1 proto tcp from any to $ip_1 port 8000 -> 192.168.0.2 port 8000
rdr pass on $wan_1 proto tcp from any to $ip_1 port 443 -> 192.168.0.2 port 443
rdr pass on $wan_1 proto tcp from any to $ip_1 port 2222 -> 192.168.0.1 port 22

rdr log on $wan_2 proto tcp from any to $ip_2 port 80 -> 192.168.0.3 port 80

pass in on $lan all
pass out all
block in on $wan_1 all
block in on $wan_2 all

pass inet proto icmp all icmp-type $icmp_types keep state

pass in on $wan_1 inet proto tcp from any to $ip_1 port $wan_1_tcp_incoming

pass in log quick on $wan_2 reply-to ( $wan_2 $ip_2 ) inet proto tcp from any to 192.168.0.3 port 80 keep state

pass in on $lan route-to ( $wan_1 $gw_1 ) from <network_1> to !<network_1_area> keep state
pass in on $lan route-to ( $wan_2 $gw_2 ) from <network_2> to !<network_1_area> keep state

pass out on $wan_1 route-to ( $wan_2 $gw_2 ) from $wan_2 to any keep state
pass out on $wan_2 route-to ( $wan_1 $gw_1 ) from $wan_1 to any keep state

pf.log output:
Code: 
00:00:00.249600 rule 10..16777216/0(match): pass in on ste1: aa.bb.cc.dd.1739 > 192.168.0.3.80: Flags [S], seq 761507850, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
 
I've just resolved the problem. I wanted to share the result in order to help others who may read this thread.

@kpa, your reply was certainly the real answer. The problem with my rules was the state-policy. When I remove state-policy definition, everything works perfectly.

@DutchDaemon, reply-to should contain the gateway IP and packets should go to the target IP as it shown on @kpa's post.

Thank you all who participated this thread to help me with sharing their valuable knowledge and time.
 
Last edited by a moderator:
My reply-to rules work just fine, so I 'should' not do anything, I believe. Perhaps you misunderstood my pf.conf line. Here it is translated:

Code: 
pass in quick on bge0 reply-to (bge0 10.10.0.1) inet proto udp from any to 10.10.0.13 port = domain keep state

The server itself has 10.10.0.13 as its IP address, and 10.10.0.1 is its gateway. This is how packets are statefully returned to a non-default gateway
 
@DutchDaemon,

I'm deeply sorry if I hurt you accidentally. I didn't talked about your ruleset and I've never said you should do anything. What I've said is "it should be like that for my ruleset to work in this case".

Thanks.
 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top