How to implement "symmetric routing enforcement" with PF in FreeBSD 8.2

[Xosted]

Hi,

I have a server with multiple NICS (em0 to em4) and several jails running on it (serv1 to serv4).

em0 is plugged on a router to a lan in 192.168.0.* and all the jails have IPs on this network via aliases on em0's IP.

I would like to make the jailed service serv3 available to the Internet through em3. I thought about using a rdr PF rule for this like so:

rdr pass log(all) on $ext_if3 inet proto { tcp, udp } from any to ($ext_if3) port 80 -> $serv3

The problem with this is that the packets are entering through em3 but leaving through em0 as shown by tcpdump:

00:00:00.000359 rule 0/0(match): rdr in on em3: PC_IP.51046 > SERV3_IP.80:  tcp 28 [bad hdr length 4 - too short, < 20]
00:00:00.000025 rule 0/0(match): rdr out on em0: EM3_IP.80 > PC_IP.51046:  tcp 20 [bad hdr length 12 - too short, < 20]
00:00:02.999608 rule 0/0(match): rdr out on em0: EM3_IP.80 > PC_IP.51046:  tcp 28 [bad hdr length 4 - too short, < 20]

The pf.conf man page gave me a clue about reply-to which seem to be what I want according to the description:

reply-to
The reply-to option is similar to route-to, but routes packets that
pass in the opposite direction (replies) to the specified inter-
face. Opposite direction is only defined in the context of a state
entry, and reply-to is useful only in rules that create state. It
can be used on systems with multiple external connections to route
all outgoing packets of a connection through the interface the
incoming connection arrived through (symmetric routing enforce-
ment).

But this can not be used on a rdr rule. I tried to find examples on how to deconstruct a rdr pass directive in a rdr and several pass ones (in and out), but I was unsuccessful. I never seem to get the syntax or anything else right to achieve this.

I tried to use this rule:

pass in quick log on $ext_if3 reply-to ( $ext_if3 ISP_ROUTER_ADDRESS ) proto { tcp, udp } from any to ($ext_if3) port 80 keep state

With this tcpdump reports packets going in from the PC_IP to SERV3_IP but nothing coming out and the browser times out.

Beside, the grammar indicated on the pf.conf man's page seems to suggest (if I understood properly of course) that the reply-to uses an interface name and an optional address.

route = ( "route-to" | "reply-to" | "dup-to" )
( routehost | "{" routehost-list "}" )
[ pooltype ]

routehost = "(" interface-name [ address [ "/" mask-bits ] ] ")"

But if I don't put the address, the syntax turns to be wrong.
I would appreciate a bit of help for this.

Here are the complete pf.confs:

ext_if3="em3"

serv3="IP_SERV3"

set skip on lo
set skip on em0 # I tried without the line too

scrub in all
                                                                                              
rdr pass log(all) on $ext_if3 inet proto { tcp, udp } from any to ($ext_if3) port 80 -> $serv3

pass log all

and

ext_if3="em3"

serv3="IP_SERV3"

set skip on lo
set skip on em0 # I tried without the line too

scrub in all
                                                                                              
rdr on $ext_if3 inet proto { tcp, udp } from any to ($ext_if3) port 80 -> $serv3

pass in quick log on $ext_if3 reply-to ( $ext_if3 ISP_ROUTER_ADDRESS ) proto { tcp, udp } from any to ($ext_if3) port 80 keep state

pass log all

 

[svl]

Try this:

rdr on $ext_if3 inet proto { tcp, udp } from any to ($ext_if3) port 80 [B]tag SERV3[/B] -> $serv3

pass in quick log on $ext_if3 reply-to ( $ext_if3 ISP_ROUTER_ADDRESS ) [B]tagged SERV3[/B]