tcp reset on reply to nat'd address

D

dove

I have ipfw running on a freebsd box with two NICs.

The "external" NIC is really facing a dumb router that is
configured to just pass everything through. That subnet is 192.168.0/24.

The "internal" subnet is actually a routable real internet
block, let's say a.b.c/24.

The firewall has its "internal" IP set to a.b.c.d on
interface em1.
"external" IP set to 192.168.0.2 with the dumb router
at 192.168.0.1 on interface em0.

I also have some internal hosts that are unregistered RFC
1918 addresses. I am trying to NAT these IPs and leave
the routable registered IPs untouched.

Rules:
Code: 
extif=em1
intif=em0
pubnet=[I]a.b.c[/I]
privnet=10.0.0.0/8
ipfw deny log ip from 192.168.0.0/24 to any in recv $intif
ipfw add divert natd log all from $privnet to any out via $extif
ipfw add allow tcp from any to [I]a.b.c.e[/I] 22 via $extif
ipfw add deny log tcp from any to any in recv $extif setup
ipfw add allow tcp from any to any setup
ipfw add allow tcp from any to any established
ipfw add allow udp 53 to [I]a.b.c[/I].0/24
ipfw add 65500 deny log ip from any to any
and natd running as [cmd=]natd -v -n em0[/cmd]

tcpdump on both interfaces looking at external host 'E.F.G.H' is showing this when I try to ssh to it from the internal 10.0.0.2 box:
Code: 
16:52:16.207810 10.0.0.2.45642 > [I]E.F.G.H[/I].22: S 3095885078:3095885078(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 288974989 0,sackOK,eol> (DF)
16:52:16.209071 [I]a.b.c.d[/I].45642 > [I]E.F.G.H[/I].22: S 3095885078:3095885078(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 288974989 0,sackOK,eol> (DF)
16:52:16.271086 [I]E.F.G.H[/I].22 > [I]a.b.c.d[/I].45642: S 1381661810:1381661810(0) ack 3095885079 win 5792 <mss 1380,nop,nop,timestamp 2738478757 288974989,nop,wscale 2> (DF)
16:52:16.271214 [I]a.b.c.d[/I].45642 > [I]E.F.G.H[/I].22: R 3095885079:3095885079(0) win 0

As you can see the rewrite is done correctly on the outgoing divert, and the external host responds. But the firewall is generating a tcp reset.

It's as if it has forgotten about the diverted socket and doesn't have an open socket for it anymore.

Hints would be wonderful, please.
 
I'm a bit out of the loop wrt ipfw, but I seem to be missing stateful rules here, which would explain why return traffic is refused. Maybe statefulness is built in/implied now (like in pf), in which case I wasn't here ;) I'm not sure whether the 'established' rule works on its own like that, i.e. without check-state and keep-state somewhere else.
 
I tried adding keep-state to the 'divert' rule (just appended it to the line) and check-state on the line after it. natd -v is now reporting an 'In' rewrite, but the tcp reset happens exactly the same as before.

Where's the right place to put the *state directives?
 
Yeah, I've been over the man page (and various howtos and blogs). I'm not finding the right magic, and could use a hint about what I'm doing wrong and/or an explanation of why the reset is happening. The handbook has an example, but the entire internal net is nat'd. In my case, I only want the RFC 1918 addresses nat'd, not the public ones. Hopefully someone can interpret the tea leaves here.
 
Thanks for the ideas.
Moving check-state to the top (or anywhere I tried it) and adding keep-state to all the allow rules didn't help. I still get the tcp reset.
 
Please post an ascii network diagram (put
Code: 
 tags around it). Your 'external' and 'internal' networks, IPs, and NICs confuse the hell out of me .. You appear to have no routing for the 10/8 network, and to be NAT'ing that network without it being directly connected to (present on) any of your interfaces. One normally NAT's a network on interface_a to interface_b. You could try adding a static route to network 10/8, pointing to the interface it's behind.
 
DutchDaemon said:
Please post an ascii network diagram (put
Code: 
 tags around it). Your 'external' and 'internal' networks, IPs, and NICs confuse the hell out of me .. You appear to have no routing for the 10/8 network, and to be NAT'ing that network without it being directly connected to (present on) any of your interfaces. One normally NAT's a network on interface_a to interface_b. You could try adding a static route to network 10/8, pointing to the interface it's behind.[/QUOTE]

Here you go.  Hopefully this will give a better picture in addition to the previous posts.  I didn't describe the internal routing for 10/8 before, but I now show that here.

When the 10.0.0.2 hits the 1.2.3.4 box (the ipfw firewall), that's when I want the NAT to happen so that I can, for instance, browse the internet or ssh or whatever from 10.0.0.2.

[CODE]
10.0.0.2 = host on private internal lan
1.2.3.4  = internal interface on ipfw firewall box; a publicly
          routable "real" internet address
10.0.0.1 = 10/8 interface on an internal router (omitted
           in previous examples since it's not directly relevant,
           but it's good for the whole picture)
1.2.3.5  = "other" interface on internal router attached
           on internal LAN with the "real" IP addresses
192.168.0.2 = "external" facing interface on firewall
192.168.0.1 = other end of the crossover cable from
              the firewall - it's a dumb router / csu configured
              to pass everything through to the firewall on 
              192.168.0.2 and has the connection to the ISP
              on the other end; ISP is configured to route
              all of 1.2.3/24 to this connection.

In the previous examples,

Code: 
a.b.c.d = 1.2.3.4
E.F.G.H = 5.6.7.8
and
a.b.c.e = 1.2.3.10  (some other internal host that allows incoming ssh)

Code: 
---------     --------------------
10.0.0.2 | - | 10.0.0.1   1.2.3.5 |
---------     --------------------
                            |
                            |         ------------
                            |--------|  1.2.3.10  |
                            |         ------------
                            |
                        ------------------------
                       | 1.2.3.4    192.168.0.2 |  [1]
                        ------------------------
                                       |
                                       |
                                    --------------------------
                                   | 192.168.0.1              |
                                   |    CSU connection to ISP |
                                    --------------------------
                                                  |
                                                  |
                                              ISP & internet
                                                 |
                                                 |
                                          ---------------------
                                         |  some external box  |
                                         |  at, say, 5.6.7.8   |
                                          ---------------------

[1] This is the IPFW firewall box
Click to expand...
 
Can you ping (or otherwise connect to) 10.0.0.2 from the IPFW box? If 1.2.3.4 sends a tcp-reset, it means it doesn't know how to handle the connection back to the 10/8 network.
 
Yes. Before I tried adding the NAT, I had rules to stop all RFC 1918 addresses from going beyond the firewall. And those deny rules (logged) are hit.

Here's the traceroute...
Code: 
% traceroute -n 10.0.0.2
traceroute to 10.0.0.2 (10.0.0.2), 64 hops max, 44 byte packets
 1  1.2.3.5  0.361 ms  0.265 ms  0.247 ms
 2  10.0.0.2  0.734 ms  0.729 ms  0.750 ms

It's the rules I need help with I think (or natd args?). If I do need keep-state, I'm unclear where to put it (and check-state) in my rule set previously shown. As I said, I tried keep-state on the divert rule and then separately on all the allow rules, with check-state at various places with no luck.

Here is the rule set again with rule numbers so it's easier to reference them in this thread and the fake IP numbers from the ASCII diagram plugged in instead of symbolic letters...


Code: 
extif=em1
intif=em0
pubnet=1.2.3
privnet=10.0.0.0/8
ipfw deny 100 log ip from 192.168.0.0/24 to any in recv $intif
ipfw add 200 divert natd log all from $privnet to any out via $extif
ipfw add 300 allow tcp from any to 1.2.3.10 22 via $extif
ipfw add 400 deny log tcp from any to any in recv $extif setup
ipfw add 500 allow tcp from any to any setup
ipfw add 600 allow tcp from any to any established
ipfw add 700 allow udp 53 to 1.2.3.0/24
ipfw add 65500 deny log ip from any to any
 
Clarification...
I said the RFC 1918 deny rules "are hit". I meant to say "were hit" (when I had them in there). In the interest of simplicity to isolate this reset problem, I pulled those rules (and some others) out.
 
Try to log every IPFW rule including the final "deny all". Sometimes it's difficult to see a little issue in the system you've built. IPFW will show you how the packet comes in, how it comes out, wether it's blocked by some rules or anything else
 
As you can see in the rules, all deny entries are logged. None of them are hit when the tcp reset occurs. I even added logging for all 'allow' entries briefly and there was no interesting information there (incoming reply from the external host is logged, and there is no deny thereafter).
 
For simplicity's sake, it might be better to do your NAT on 10.0.0.1.

The problem with your current setup is your divert rule. It is only diverting traffic that exits $extif. You need to remember that natd needs to do the aliasing of outgoing traffic and the dealiasing of incoming traffic. You are not passing incoming traffic back through natd to dealias it, so ipfw is rejecting it.
 
Okay, let's reformulate the question to do it on 10.0.0.1 then.

What would be the rules if I wanted to NAT only traffic to and from 10.0.0.2 and none of the other hosts on 10/8?
 
dove said:
What would be the rules if I wanted to NAT only traffic to and from 10.0.0.2 and none of the other hosts on 10/8?
Click to expand...
Something like:

Code: 
ipfw add divert natd from 10.0.0.2 to any out via $extif
ipfw add divert natd from any to me in via $extif
 
You must log in or register to reply here.
Back
Top