Forward HTTP traffic to internal web servers

samsam9988

samsam9988

Hi, I own two public address x.x.x.x and y.y.y.y for my two web servers. Another z.z.z.z public address is assigned to tun0. A FreeBSD 10 system is configured as a gateway to forward traffic from tun0 to the two internet web servers. Illustrated as below:
Code: 
x.x.x.x
z.z.z.z -- tun0 ---- 2 web servers (192.168.5.20 via em2 and 192.168.6.20 via em3)
y.y.y.y
However when I browsed x.x.x.x from my iPhone (outside my local network), traffic is not getting through to the web servers. Same as when I browsed y.y.y.y.I am wondering what should I do to make this happen?

My rc.conf configuraiton:
Code: 
gateway_enable="YES"

inetd_enable="YES"
kern_securelevel_enable="NO"

hostname="iExtentGateway"
#ifconfig_em0="DHCP"
#ifconfig_em4="inet 192.168.1.254 netmask 255.255.255.0"
#ifconfig_em4="inet 10.0.1.254 netmask 255.255.255.0"
ifconfig_em0="inet 192.168.2.254 netmask 255.255.255.0"
ifconfig_em1="inet 192.168.1.254 netmask 255.255.255.0"
ifconfig_em2="inet 192.168.5.254 netmask 255.255.255.0"
ifconfig_em3="inet 192.168.6.254 netmask 255.255.255.0"

ppp_enable="yes"
ppp_profile="pppoe"
ppp_mode="ddial"


natd_enable="YES"
natd_flags="-f /etc/natd.conf"
firewall_enable="YES"
firewall_logging="YES"
firewall_script="/etc/ipfw.rules"

natd.conf file:
Code: 
interface tun0
use_sockets yes
dynamic yes
redirect_address 192.168.5.20 x.x.x.x
redirect_address 192.168.6.20 y.y.y.y

ipfw.rules file:
Code: 
#!/bin/sh

IPF="ipfw -q add"
ipfw -q -f flush
skip="skipto 800"

$IPF divert natd all from any to any via tun0

#loopback
$IPF 10 allow all from any to any via lo0
$IPF 20 deny all from any to 127.0.0.0/8
$IPF 30 deny all from 127.0.0.0/8 to any
$IPF 40 deny tcp from any to any frag

# statefull
$IPF 50 check-state
$IPF 60 allow tcp from any to any established
$IPF 70 allow all from any to any out keep-state
$IPF 80 allow icmp from any to any
# open port ftp (20,21), ssh (22), mail (25)
# http (80), dns (53) etc
$IPF 130 allow tcp from any to any 22 in
$IPF 140 allow tcp from any to any 22 out
$IPF 150 allow tcp from any to any 25 in
$IPF 160 allow tcp from any to any 25 out
$IPF 170 allow udp from any to any 53 in
$IPF 175 allow tcp from any to any 53 in
$IPF 180 allow udp from any to any 53 out
$IPF 185 allow tcp from any to any 53 out
$IPF 200 allow tcp from any to any 80 in
$IPF 210 allow tcp from any to any 80 out
$IPF 720 allow all from 192.168.0.0/16 to any keep-state
$IPF 722 allow all from 10.0.0.0/8 to any keep-state
$IPF 730 allow tcp from any to x.x.x.x 80 keep-state
$IPF 740 allow tcp from any to y.y.y.y 80 keep-state

# deny and log everything
$IPF 900 deny log all from any to any
Very appreciate suggestion and help.
Best regards
Sam
 
samsam9988 said:
However when I browsed x.x.x.x from my iphone (outside my local network), traffic is not getting through to the web servers. Same as when I browsed y.y.y.y
Click to expand...
Is this traffic actually routed to your FreeBSD host? Does it even arrive there?
 
SirDice said:
samsam9988 said:
However when I browsed x.x.x.x from my iphone (outside my local network), traffic is not getting through to the web servers. Same as when I browsed y.y.y.y
Click to expand...
Is this traffic actually routed to your FreeBSD host? Does it even arrive there?
Click to expand...
Yes it is arriving tun0:

Code: 
root@iExtentGateway:/etc # tcpdump -i tun0 dst y.y.y.y
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type NULL (BSD loopback), capture size 65535 bytes
01:16:48.781904 IP a12-09-03.opera-mini.net.48780 > y.y.y.y.static.exetel.com.au.http: Flags [S], seq 3633602986, win 14600, options [mss 1440,sackOK,TS val 2444293098 ecr 0,nop,wscale 7], length 0

^C
4 packets captured
2315 packets received by filter
0 packets dropped by kernel
root@iExtentGateway:/etc # tcpdump -i tun0 dst x.x.x.x
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type NULL (BSD loopback), capture size 65535 bytes
01:17:14.332614 IP a12-09-03.opera-mini.net.52264 > x.x.x.x.static.exetel.com.au.http: Flags [S], seq 4109861018, win 14600, options [mss 1440,sackOK,TS val 2444299486 ecr 0,nop,wscale 7], length 0
 
samsam9988 said:
SirDice said:
samsam9988 said:
However when I browsed x.x.x.x from my iphone (outside my local network), traffic is not getting through to the web servers. Same as when I browsed y.y.y.y
Click to expand...
Is this traffic actually routed to your FreeBSD host? Does it even arrive there?
Click to expand...
Yes it is arriving tun0:

Code: 
root@iExtentGateway:/etc # tcpdump -i tun0 dst y.y.y.y
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type NULL (BSD loopback), capture size 65535 bytes
01:16:48.781904 IP a12-09-03.opera-mini.net.48780 > y.y.y.y.static.exetel.com.au.http: Flags [S], seq 3633602986, win 14600, options [mss 1440,sackOK,TS val 2444293098 ecr 0,nop,wscale 7], length 0

^C
4 packets captured
2315 packets received by filter
0 packets dropped by kernel
root@iExtentGateway:/etc # tcpdump -i tun0 dst x.x.x.x
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type NULL (BSD loopback), capture size 65535 bytes
01:17:14.332614 IP a12-09-03.opera-mini.net.52264 > x.x.x.x.static.exetel.com.au.http: Flags [S], seq 4109861018, win 14600, options [mss 1440,sackOK,TS val 2444299486 ecr 0,nop,wscale 7], length 0
Click to expand...

I tried to add the following rules in /etc/ipfw.rules:
Code: 
00200 divert natd ip from x.x.x.x 80 to 192.168.5.20 dst-port 80 via tun0
00300 divert natd ip from y.y.y.y 80 to 192.168.6.20 dst-port 80 via tun0
or
Code: 
00200 divert natd ip from x.x.x.x 80 to 192.168.5.20 dst-port 80 
00300 divert natd ip from y.y.y.y 80 to 192.168.6.20 dst-port 80
But incoming HTTP traffic to x.x.x.x and y.y.y.y is not being forwarded to my internal servers.
 
I don't use IPFW but on PF the redirections are done like this:
Code: 
rdr on $ext_if inet proto tcp from any to $addr1 port 80 -> 192.168.5.20 port 80
rdr on $ext_if inet proto tcp from any to $addr2 port 80 -> 192.168.6.20 port 80

Which roughly translates as "traffic from anywhere to IP $addr2 port 80 coming in on $ext_if needs to be redirected to 192.168.5.20 port 80".
 
SirDice said:
I don't use IPFW but on PF the redirections are done like this:
Code: 
rdr on $ext_if inet proto tcp from any to $addr1 port 80 -> 192.168.5.20 port 80
rdr on $ext_if inet proto tcp from any to $addr2 port 80 -> 192.168.6.20 port 80

Which roughly translates as "traffic from anywhere to IP $addr2 port 80 coming in on $ext_if needs to be redirected to 192.168.5.20 port 80".
Click to expand...

PF in FreeBSD 10 is deadly slow. It becomes useless.
that s I switched back to IPFW.
 
samsam9988 said:
PF in FreeBSD 10 is deadly slow. It becomes useless.
Click to expand...
I doubt it's slow as FreeBSD 10 had a major update on PF to be able to use SMP more efficiently. But there could of course be bugs. Have you tried 10-STABLE or the 10.1-PRERELEASE?
 
samsam9988 said:
SirDice said:
I don't use IPFW but on PF the redirections are done like this:
Code: 
rdr on $ext_if inet proto tcp from any to $addr1 port 80 -> 192.168.5.20 port 80
rdr on $ext_if inet proto tcp from any to $addr2 port 80 -> 192.168.6.20 port 80

Which roughly translates as "traffic from anywhere to IP $addr2 port 80 coming in on $ext_if needs to be redirected to 192.168.5.20 port 80".
Click to expand...

PF in FreeBSD 10 is deadly slow. It becomes useless.
that s I switched back to IPFW.
Click to expand...

Care to share your setup and explain your testing methods for the "slowness". This is about the first time I've seen anyone claim PF on FreeBSD 10 as slow.
 
kpa said:
samsam9988 said:
SirDice said:
I don't use IPFW but on PF the redirections are done like this:
Code: 
rdr on $ext_if inet proto tcp from any to $addr1 port 80 -> 192.168.5.20 port 80
rdr on $ext_if inet proto tcp from any to $addr2 port 80 -> 192.168.6.20 port 80

Which roughly translates as "traffic from anywhere to IP $addr2 port 80 coming in on $ext_if needs to be redirected to 192.168.5.20 port 80".
Click to expand...

PF in FreeBSD 10 is deadly slow. It becomes useless.
that s I switched back to IPFW.
Click to expand...

Care to share your setup and explain your testing methods for the "slowness". This is about the first time I've seen anyone claim PF on FreeBSD 10 as slow.
Click to expand...

Sorry in FreeBSD 9.2-RELEASE.
 
PF in FreeBSD 10 is deadly slow. It becomes useless.
that s I switched back to IPFW.
Click to expand...

Care to share your setup and explain your testing methods for the "slowness". This is about the first time I've seen anyone claim PF on FreeBSD 10 as slow.
Click to expand...

sorry in Freebsd 9.2-Release
Click to expand...

The setup in FreeBSD 9.2 release is as shown below:
Code: 
root@iExtentGateway:/etc # cat pf.conf
#       $FreeBSD: release/9.2.0/share/examples/pf/pf.conf 218854 2011-02-19 14:57:00Z brucec $
#       $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $
#
# See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
# Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1
# in /etc/sysctl.conf if packets are to be forwarded between interfaces.

tcp_services = "{http, ssh, imaps, smtp, domain, ntp, www, https, 18080, 1444}"
udp_services= "{domain, ntp}"

#ext_if="em4"
ext_if="tun0"
int_if="em0"
webservice_if="em1"
seacell_ip="x.x.x.x"
morfeus_ip="y.y.y.y"
iextserv_ip="192.168.1.20"
gateway_ip="z.z.z.z"
seacell_if="em2"
morfeus_if="em3"
iextserv_if="em1"

#table <spamd-white> persist

set skip on lo

scrub in

nat on $ext_if from !($ext_if) -> ($ext_if:0)

rdr pass on $ext_if proto tcp from any to $seacell_ip port 18080 -> 192.168.1.20 port 18080
rdr pass on $ext_if proto tcp from any to $morfeus_ip port 18080 -> 192.168.1.20 port 18080
# for jboss webservice runtime.... keep this redirection even for go live...
rdr pass on $ext_if proto tcp from any to $seacell_ip port 14447 -> 192.168.1.20 port 14447
rdr pass on $ext_if proto tcp from any to $morfeus_ip port 14447 -> 192.168.1.20 port 14447

rdr pass on $ext_if proto tcp from any to $seacell_ip port 80 -> 192.168.5.20 port 80
rdr pass on $ext_if proto tcp from any to $morfeus_ip port 80 -> 192.168.6.20 port 80
rdr pass on $ext_if proto tcp from any to $seacell_ip port 22 -> 192.168.5.20 port 22 
rdr pass on $ext_if proto tcp from any to $morfeus_ip port 22 -> 192.168.6.20 port 22
rdr pass on $ext_if proto tcp from any to $seacell_ip port 53 -> 192.168.5.20 port 53 
rdr pass on $ext_if proto tcp from any to $morfeus_ip port 53 -> 192.168.6.20 port 53 
rdr pass on $ext_if proto udp from any to $seacell_ip port 53 -> 192.168.5.20 port 53 
rdr pass on $ext_if proto udp from any to $morfeus_ip port 53 -> 192.168.6.20 port 53 
rdr pass on $ext_if proto icmp from any to $seacell_ip -> 192.168.5.20 
rdr pass on $ext_if proto icmp from any to $morfeus_ip -> 192.168.6.20

antispoof quick for $ext_if

#anchor "ftp-proxy/*"
block in log
pass out quick modulate state

#pass everything for internal server and pcs:
pass quick on $webservice_if modulate state
pass quick on $int_if modulate state

#antispoof quick for { lo $int_if }

#pass incoming www trafic:
pass quick on $ext_if proto tcp from any to any port $tcp_services modulate state
pass quick on $ext_if proto udp from any to any port $udp_services modulate state


pass quick on $seacell_if from any to any modulate state
pass quick on $morfeus_if from any to any modulate state
pass quick on $iextserv_if from any to any modulate state

#pass incoming ssh:
pass in quick on $ext_if proto tcp to ($ext_if) port ssh modulate state

#pass in log on $ext_if proto tcp to ($ext_if) port smtp

#pass out smtp:
pass out log on $ext_if proto tcp from ($ext_if) to port smtp
pass out log on $seacell_if proto tcp from ($seacell_if) to port smtp
pass out log on $morfeus_if proto tcp from ($morfeus_if) to port smtp

#pass incoimg pings:
pass in on $ext_if inet proto icmp from any to ($ext_if) icmp-type { unreach, redir, timex }
pass in on $seacell_if inet proto icmp from any to ($seacell_if) icmp-type { unreach, redir, timex }
pass in on $morfeus_if inet proto icmp from any to ($morfeus_if) icmp-type { unreach, redir, timex }
The slowness is happen gradually.
 
What kind of bandwidth usage and how many concurrent connections are there? If you have a lot of connections you may simply be running out of memory for the state-tables. The slowness may also be caused by the VPN. What are you using for VPN?
 
SirDice said:
What kind of bandwidth usage and how many concurrent connections are there? If you have a lot of connections you may simply be running out of memory for the state-tables. The slowness may also be caused by the VPN. What are you using for VPN?
Click to expand...

Nah, nothing to do with my enriovvironment. I am running IPFW, it is super fast.
 
samsam9988 said:
PF may be can't handle "so-many" network interfaces.
Click to expand...
Can anyone suggest a working example for redirecting incoming HTTP traffic to an internal web server?

I tried to add the following rule in IPFW:

Code: 
$CMD divert natd all from any to x.x.x.x 80 dst 192.168.5.20 80
or
Code: 
$CMD divert natd all from an to x.x.x.x 80 redivert-to 192.168.1.5.20 80

This is only a sudopseudo-code rather than a working solution. I don't know what the exact IPFW syntax for doing that.

Best regards
Sam
 
samsam9988 said:
...
natd.conf file:
Code: 
interface tun0
use_sockets yes
dynamic yes
redirect_address 192.168.5.20 x.x.x.x
redirect_address 192.168.6.20 y.y.y.y
Click to expand...

Reading the manual of natd(8), it seems to me that the interface directive in your natd.conf does not exactly what you intend it to do, namely natting all traffic that passes by tun0.

http://www.freebsd.org/cgi/man.cgi?query=natd&sektion=8&n=1 said:
-interface | -n interface
Use interface to determine the aliasing address. If there is
a possibility that the IP address associated with interface
may change, the -dynamic option should also be used. If this
option is not specified, the -alias_address option must be
used.

The specified interface is usually the ``public'' (or
``external'') network interface.
Click to expand...

Above means to me clearly, that in your setup, only the traffic from/to z.z.z.z is natted, which is not what you want.

I am not able to test the following, since I got only one public IP, and beside that, I use the NAT that is builtin to ipfw(8). Anyway, at least it should give you an idea:
Code: 
use_sockets yes
dynamic yes

# default instance
# public1
port 8668
alias_address x.x.x.x
redirect_address 192.168.5.20 x.x.x.x

# second instance
instance public2
port 8888
alias_address y.y.y.y
redirect_address 192.168.6.20 y.y.y.y
 
samsam9988 said:
I have 5
Click to expand...
So you are saying if I have 5 the performance is going to be unacceptable?
You didn't said what kind of hardware do you have and why won't you try FreeBSD 10 as pf have major optimization there?
Maybe someone who have 5 or more ports can share his experience with pf, I am curious about this ;)
 
nforced said:
samsam9988 said:
I have 5
Click to expand...
So you are saying if I have 5 the performance is going to be unacceptable?
You didn't said what kind of hardware do you have and why won't you try FreeBSD 10 as pf have major optimization there?
Maybe someone who have 5 or more ports can share his experience with pf, I am curious about this ;)
Click to expand...
Code: 
# ifconfig -akk
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 00:15:17:f2:e5:91
        inet 192.168.2.254 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fe80::215:17ff:fef2:e591%em0 prefixlen 64 scopeid 0x1 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 00:15:17:f2:e5:90
        inet 192.168.1.254 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::215:17ff:fef2:e590%em1 prefixlen 64 scopeid 0x2 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 00:15:17:f2:e5:93
        inet 192.168.5.254 netmask 0xffffff00 broadcast 192.168.5.255
        inet6 fe80::215:17ff:fef2:e593%em2 prefixlen 64 scopeid 0x3 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether 00:15:17:f2:e5:92
        inet 192.168.6.254 netmask 0xffffff00 broadcast 192.168.6.255
        inet6 fe80::215:17ff:fef2:e592%em3 prefixlen 64 scopeid 0x4 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 44:87:fc:73:c8:f7
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
 
You must log in or register to reply here.
Back
Top