IPFW How to prevent port scanning

A

akshin

Please help me.
This is part of tail -f /var/log/security. Start scanning ports from 0 to 65535. Because of this, the web server slows down.
nginx load balancing.
10.10.10.22 - proxy server(nginx)
10.10.10.55:9090 - proxied server(nginx)
10.10.10.90:9090 - proxied server(nginx)
Code: 
Mar 16 18:49:50 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:43031 in via vmx0
Mar 16 18:49:54 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:42214 in via vmx0
Mar 16 18:49:55 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45267 in via vmx0
Mar 16 18:49:56 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45810 in via vmx0
Mar 16 18:49:56 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45778 in via vmx0
Mar 16 18:49:56 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45840 in via vmx0
Mar 16 18:49:57 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:43209 in via vmx0
Mar 16 18:49:59 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45926 in via vmx0
Mar 16 18:50:01 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45641 in via vmx0
Mar 16 18:50:06 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:55189 in via vmx0
Mar 16 18:50:10 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:42263 in via vmx0
Mar 16 18:50:11 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45063 in via vmx0
Mar 16 18:50:11 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45810 in via vmx0
Mar 16 18:50:11 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:43118 in via vmx0
Mar 16 18:50:12 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45778 in via vmx0
Mar 16 18:50:12 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45840 in via vmx0
Mar 16 18:50:14 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:55231 in via vmx0
Mar 16 18:50:14 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:42809 in via vmx0
Mar 16 18:50:15 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45926 in via vmx0
Mar 16 18:50:16 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:42554 in via vmx0
Mar 16 18:50:19 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:43195 in via vmx0
Mar 16 18:50:21 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.55:9090 10.10.10.22:45267 in via vmx0
Mar 16 18:50:27 cluster kernel: ipfw: 65534 Deny TCP 10.10.10.90:9090 10.10.10.22:45778 in via vmx0

I want to know what to do. What to check and how to check.
Help me please.
 
These are all internal addresses. So something's up on your own network. And these don't look like a typical scan, they look like responses being blocked (responses to connections on port 9090). So, what's running on 10.10.10.22 and is trying to connect to 10.10.10.90 and 10.10.10.55 on port 9090? What's running on port 9090?
 
Why start scanning ports from 0 to 65535.
Running nginx php-fpm mysql spinxsearch redis memcached
 
akshin said:
Why start scanning ports from 0 to 65535.
Click to expand...
It doesn't. Normal connections always use a random source port. So what you are seeing are the responses to multiple connections, each using a different source port. This is normal.

akshin said:
may be some bot infected into server?
Click to expand...
I doubt that. Check your logs on that 10.10.10.22 machine, something may be hammering your proxy.
 
this from sockstat -4 on 10.10.10.90

Code: 
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45460
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45465
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45466
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45467
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45469
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45490
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45472
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45474
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45422
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45479
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45480
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45481
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45484
?        ?          ?     ?  tcp4   10.10.10.90:9090        10.10.10.22:45181

What is ? ? ? ? How to know USER COMMAND PID FD
 
Code: 
netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address          Foreign Address        (state)    
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65022        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65019        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65018        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65016        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65013        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65009        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65004        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.65002        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64997        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64996        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64993        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64991        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64987        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64985        TIME_WAIT  
tcp4       0      0 10.10.10.90.9090         10.10.10.22.64977        TIME_WAIT
 
There is a couple of systems management software suites commonly living on port tcp:9090.
So if You own the 10.10.10.90, then why don't You just look what's sitting there?

Ah, sysutils/lsof is able to tell which process uses a given port. There should also be a command in the base system, but I forget the name of that. Maybe somebody can help out?
 
I think the question marks show a connection for a process that's finished - so it can't tell you anything about the process or the user who started it.

You could wrong sockstat every second (or less) and log the output and see if you can catch the change of a process from a number to a question mark.

Might be some short-lived process making a lot of requests.
 
Have you checked the nginx logs on 10.10.10.22 yet? The sockstat(1) outputs aren't that interesting to look at. They only provide some indication something's not right but don't provide any more information that will allow you to ascertain what's going on. They are most likely a symptom caused by something else.
 
akshin said:
I checked all the logs mysql nginx php. There is nothing about it.
Click to expand...
Post them any way. You probably don't know what to look for. And post your proxy configuration from the 10.10.10.22 machine.
 
You must log in or register to reply here.
Back
Top