Can't get inside my LAN from outside, who's to blame?

Hi guys, this is a tough one but I figure you can get it sorted.

I am trying to either port forward or DMZ a server box. I can get from anywhere in my LAN to anywhere, all my boxes are set up with SSH pubkey auth. However, I cannot get inside my LAN from outside, all I get is "connection refused".

The ISP and DSL provider both swear they aren't blocking any ports. Each one naturally blames the other one so no help from there.

When I DMZ the box I want to reach from outside, canyouseeme.org reports that port 22 is open. But if I run nmap inside my LAN it says it's open but from outside it's closed. How is this possible?!

I have no idea to troubleshoot this, but I would like to prove my router is screwing the works and I don't have another router to try. I suspect the router because when I telnet from inside my lan to ports 23 or 80 on my external IP address I get the router's management interface rather than the connection refused I would expect since I'm not running a webserver on the server box. And nmap says the port is closed and I trust it a lot more than I trust some website.

Any ideas? Thanks!
 
Most routers don't like it when you scan the outside address from inside the network. Things go really weird when you're forwarding ports from outside back in.

Same goes for trying to connect anything to your outside address. To test this correctly you have to do the tests also from outside your network, not from within.

Getting to grips with this only works if you have a clear understanding of how TCP/IP works. Your post shows you know some basic things but not enough. For starters learn what a three-way-handshake is and how this relates to TCP connections. That should probably clear up a few things.
 
randux said:
when I telnet from inside my lan to port 80 on my external IP address I get the router's management interface rather than the connection refused I would expect since I'm not running a webserver on the server box.

This is most likely because your outside IP address is assigned to your router, not your server. You are going to the web server on the router, which is why you get the config page. You will also not get very good results trying to ssh from the inside to the outside and back in. You should do your testing from outside your network. Think of it like trying to get from your bedroom to your bathroom by forcing yourself to go through the front door.

You need to forward port 22 from your router's external address to your server's internal address (192.168.x.x or 10.x.x.x or whatever).
 
I already tried port forwarding and DMZ as I said. I can understand SirDice's comment about perhaps not being able to test from inside, but I think nmap is giving me a pretty clear proof the router is grabbing the port and not forwarding it.

Any other helpful suggestions aside from pointing out I'm not a network guy? That's obvious, or I wouldn't be asking these questions.
 
Run the nmap from the outside, not from your own LAN. As others have pointed, trying to connect (or do any kind of portscan) to the public ip address that your router has on its external interface from your LAN may not work at all or may give funny results.
 
Is there any online service to do this? I don't know anyone to ask.

edit: Found one and am trying it now.
 
Does your ISP allow you to ssh into their servers? Mine does and it's great for testing :e
 
If I sign up for a shell account somewhere will this help me test these types of problems or will that not work either?
 
SirDice said:
Does your ISP allow you to ssh into their servers? Mine does and it's great for testing :e

So I would ssh in there and then ssh to my test host? I don't know, but I'll ask. Right now they're saying if canyouseeme.org works then it's not their problem. I need to be able to prove nothing is coming into my server, but I don't know how to capture and find any incoming traffic. Thanks.

edit: even if I get a connection refused when I ssh from somewhere else, it won't prove anything (although AFAIK you get a "denied publickey" if your key doesn't match. The ISP and DSL companies say my server is broken but that doesn't make sense since I can ssh to it from my LAN. Could it be anything else? There isn't any hosts.deny or hosts.allow, I don't know what else on Solaris could be causing this.
 
kpa said:

Thanks, I forgot about that one. I'm trying nmap-online.com now. But grc is good.

Maybe the site I tried is bogus, it says scan ran too long and doesn't show me any output. LOL.

Off to get a shell account, anybody know any free good ones with nmap?
 
randux said:
So I would ssh in there and then ssh to my test host?
Yes, that's the idea. The ISP's box is 'outside' your network.

As for capturing traffic, use tcpdump(1):

# tcpdump -ni rl0 port 22

You should at least see the first SYN packet coming in from outside your network.
 
Thanks alot, I'll give it a try. If anyone knows a good free shell provider, please post in this thread.
 
Thanks SirDice, I have two traces here to show you.

My router is definitely not behaving 100% as it should but I cannot yet prove it's causing this. And since my test Solaris server box behaved badly since I started playing around with zones, I decided to make things simpler and created a webserver in a Linux VM and used bridged networking so I can control what IP address the router assigns, so far so good.

I am testing using a web browser on a machine with internal ip of 10.0.0.2.

When I use the router's built-in rules to port forward to my virtual webserver or DMZ the VM it doesn't work. When I type in my external IP I get the management signon for my router instead of my webserver.

When I type the local virtual webserver ip (10.0.0.8) it works normally. When I type in the external address:1080 it gets an immediate connection refused.

Then I set the virtual webserver to listen on 1080 and tried both port forwarding and DMZ and I get the same results. When I use a local ip:1080 it works, when I use my external ip:1080 it gets a connection refused.

This is the trace from the local ip to the virtual webserver listening on 1080. It works.

Code:
07:55:22.387953 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [S], seq 2604069402, win 5840, options [mss 1460,sackOK,TS val 62099185 ecr 0,nop,wscale 7], length 0
07:55:22.387979 IP 10.0.0.8.1080 > 10.0.0.2.48286: Flags [S.], seq 3426188323, ack 2604069403, win 4344, options [mss 1460,sackOK,TS val 588079 ecr 62099185,nop,wscale 5], length 0
07:55:22.388794 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [.], ack 1, win 46, options [nop,nop,TS val 62099185 ecr 588079], length 0
07:55:22.388980 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [P.], seq 1:497, ack 1, win 46, options [nop,nop,TS val 62099186 ecr 588079], length 496
07:55:22.388998 IP 10.0.0.8.1080 > 10.0.0.2.48286: Flags [.], ack 497, win 170, options [nop,nop,TS val 588080 ecr 62099186], length 0
07:55:22.392798 IP 10.0.0.8.1080 > 10.0.0.2.48286: Flags [P.], seq 1:154, ack 497, win 170, options [nop,nop,TS val 588084 ecr 62099186], length 153
07:55:22.393076 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [.], ack 154, win 54, options [nop,nop,TS val 62099190 ecr 588084], length 0
07:55:22.393083 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [F.], seq 497, ack 154, win 54, options [nop,nop,TS val 62099190 ecr 588084], length 0
07:55:22.393419 IP 10.0.0.8.1080 > 10.0.0.2.48286: Flags [F.], seq 154, ack 498, win 170, options [nop,nop,TS val 588084 ecr 62099190], length 0
07:55:22.393523 IP 10.0.0.2.48286 > 10.0.0.8.1080: Flags [.], ack 155, win 54, options [nop,nop,TS val 62099190 ecr 588084], length 0

When I hit ctrl-c I get:

Code:
10 packets captured
10 packets received by filter
0 packets dropped by kernel
------------------

This is the trace from my external ip to the virtual webserver listening on 1080. It doesn't work.

Code:
07:55:39.323010 IP 10.0.0.2.52152 > my.external.ip.addr.1080: Flags [S], seq 2873564017, win 5840, options [mss 1460,sackOK,TS val 62116122 ecr 0,nop,wscale 7], length 0
07:55:39.324060 IP my.external.ip.addr.1080 > 10.0.0.2.52152: Flags [R.], seq 0, ack 2873564018, win 0, length 0

When I hit ctrl-c I get:

Code:
2 packets captured
2 packets received by filter
0 packets dropped by kernel
 
Hmmm the plot thickens.

Using a shell account I can ssh into my virtual server and also see the virtual website. Why can't I do it using my own external ip address from inside my LAN? Does this make any sense?
 
randux said:
Why can't I do it using my own external ip address from inside my LAN?
I'll try to explain. When you connect to any outside address your packets get send to the default gateway. This is usually your router. Before leaving the external interface and travel to the internet the source address is modified to your external address (NAT). After the NAT it will look at the packet again, specifically the destination address, to see where to send it next. Now it could connect to your outside address because it's already there but you have added a forwarding rule that would send the packet back in. Some routers won't forward it and just send it out on the internet. Some routers do forward the packet back in but since the source address was already translated the response will go to your router, which has to look up the state and where the original packet came from.

It's confusing even trying to explain it, your router will certainly be confused and simply fail.
 
Thanks alot for the help. I guess I am starting to understand your earlier post about not being able to do it from inside my LAN. At first I just didn't get it, but now with your explanation and seeing the trace (although most of it was Greek to me) I realize as silly as it seems, it just won't work. Thanks everyone for the suggestions. Looks like having a shell or competent tech friends is essential to putting a box on the air and making sure it works.

note to mods: I need to make a few changes on my Solaris box and test it or I may have to create another thread so I'll resist changing this to [solved] just yet.
 
I guess we can mark this one closed. Turns out me and my router are to blame. My router doesn't port forward correctly but DMZ does seem to work. And I didn't get it that you can't test from inside your LAN even using your external IP. Look at me, I'll be a qualified network analyst in another few dozen years :p Thanks guys, cheers!
 
One thing I do when trying to figure these things out is I try to imagine I'm a network packet. Then I follow each step of the way and think about what happens to me (as the packet) and where I get send to. Networking isn't difficult, just take each step separately instead of trying to understand the whole. A network packet doesn't understand the whole either, it only knows it's next step, one step at a time.
 
randux: Rip the router out and replace it with a FreeBSD box if your FreeBSD box rungs 24/7 it may a well be the router. Most DSL (or Cable?) plastic routers can be degraded to a PPPoE modem. Use net/mpd5 as PPP implementation, setup a basic filter rules in ipfw or pf including NAT and be happy.
 
Crest said:
Most DSL (or Cable?) plastic routers can be degraded to a PPPoE modem.
Not all DSL providers support this, like mine for instance.
 
Back
Top