PF PF firewall for transparent TCP proxy

Hi there,

We have a transparent TCP proxy application for Linux (with epoll) which I ported to FreeBSD (with kqueue).
This is just a test application which transparently accepts a TCP connection from a peer, gets its source/destination ip-port and makes a connection from the same source to the same destination and after that just relay the traffic.
On Linux the acceptor socket needs to be setup with
C:
setsockopt(sk, IPPROTO_IP, IP_TRANSPARENT)
so that the destination ip-port can be later retrieved from the accepted socket simply using getsockname. The outgoing socket also needs to use IP_TRANSPARENT in order to be able to bind to non local ip-port. The iptables rules needed for this proxy to work are something like:
Code:
-A PREROUTING -p tcp -m socket -j DIVERT # If there is an opened socket for a packet send the packet to it
-A PREROUTING -i -p tcp --dport 80 -j TPROXY --on-port <port> --on-ip <ip> --tproxy-mark <mark> # Redirects all packets not consumed by the above rule to the transparent listener. Usually the first SYN and the 3rd ACK from the 3 way handshake
-A DIVERT -j MARK --set-xmark <mark>
-A DIVERT -j ACCEPT
The above test setup works fine on Linux. Now I'm trying to do the same thing on FreeBSD using IP_BINDANY and PF firewall.
I tried to redirect the incoming connections with the following rule:
rdr log on em2 inet proto tcp from any to any port 80 -> (lo0) port 8080
The problem is that I received the socket in the application with destination 127.0.0.1:8081 instead of the original destination. I checked with the pflog and saw that the rule itself seems to change the desitnation
18:37:58.823472 rule 0/0(match): rdr in on em2: 10.20.30.1.37873 > 127.0.0.1.8081: Flags [S], seq 367059445, win 64240, options [mss 1460,sackOK,TS val 2607264902 ecr 0,nop,wscale 7], length 0
Later I tried using divert-to hoping that it'll do the job for transparent redirection:
pass in log on em2 inet proto tcp from any to any port 80 divert-to localhost port 8081
and I saw the rule matching in the pflog
18:45:04.611782 rule 0/0(match): pass in on em2: 10.20.30.1.44767 > 195.85.215.215.80: Flags [S], seq 3676405420, win 64240, options [mss 1460,sackOK,TS val 2607690690 ecr 0,nop,wscale 7], length 0
but I didn't receive any socket in the application.

So my question are:
1. Is there a way to transparently redirect packets to a given socket using PF?
2. If this can't be done with PF, can it be done with IPFW?
3. Can you give me some example of firewall which would do the thing like the Linux firewall above?

Thanks,
Pavel.
 
# Squid redirect force outgouing trafic
#rdr on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 8080
 
In your example IP destination would be rewritten, while a topic starter need it untouched.
Yes. I haven't tested squid but looked at its source code and it seems to me that it receives the destination rewritten and then does additional ioctl call to /dev/pf to get the original destination.

C:
struct pfioc_natlook nl;                                                   
static int pffd = -1;                                                       
...                                                             
pffd = open("/dev/pf", O_RDONLY);
...
 if (ioctl(pffd, DIOCNATLOOK, &nl)) {
...
I'm going to try with IPFW. It seems that squid doesn't do anything additional for the case of IPFW.
 
I'll just leave these notes as a reference here if someone needs to do the same thing.
I was finally able to do it with IPFW instead of PF using the following two rules:

TLDR:
Code:
ipfw -q add 00110 fwd 127.0.0.1,8080 tcp from not me to any 80 recv ixl0
ipfw -q add 00111 fwd 127.0.0.1 tcp from any 80 to not me recv ixl0

Longer explanation:
Let's use the following scheme: LAN <-> transparent proxy <-> WAN (port 80 servers).
The first rule ensures that the packets from the LAN connections are forwarded correctly to the listener socket 127.0.0.1:8080 (SYN packet and last ACK from the 3 way handshake) or to the already opened sockets but only for LAN connections.
The second rule ensures that the packets from the WAN connections are forwarded correctly to the opened sockets for WAN connections.

Some insights after walking through the FreeBSD networking code.
The opened sockets are always with IPs which are not local for the machine. A key thing for the redirection to them to work is the forwarding to some local IP. This local IP is stored as a next hop internally. This local next hop allows the packet to be accepted for local processing rather than forwarded (this decision is made in netinet/ip_input.c). The next key thing is that in the TCP layer (netinet/tcp_input.c) the code first search for a socket using the source and destination ip-port pairs from the packet and if this fails it uses the ip-port of the next hop. This ensures that packets for already opened sockets will be queued to them instead of trying to queue them to the next hop (which doesn't even exists for the second rule - nobody listens there).
 
Back
Top