Solved Server stops responding to client

[stoe]

Dear Forum

I have issues with a FreeBSD server; As I cannot analyze or fix the problem any further, I kindly ask you for hints as to where to look for possible causes.

The server is located in my subnet 2 (sn2, 10.1.2.0/24):

root@server:~ # uname -a
FreeBSD server 12.0-RELEASE-p9 FreeBSD 12.0-RELEASE-p9 GENERIC  amd64

It is configured with a static IPv4 address and only contains one jail that is running a web application on a FAMP stack. In general it runs pretty well and is accessible to outside users as well. There are no other services running on the server and no other hosts on sn2.

Now it has happened twice that (two different) clients from my subnet 1 (sn1, 10.1.1.0/24) lost their connection to the server while working with the web application. That means the browser couldn’t connect to the server’s IP address any more; it could not even connect to the server over ssh or ping it any more, pings resulted in infinite request timeouts.
NB: These were two separate incidents, not happening at the same time, but the symptoms were identical and I guess the cause was as well.

As the affected client could still ssh to other hosts on sn1 and to the router (10.1.1.1), I guess it’s not the client that is causing this issue.
As other clients on sn1 (including the router) or the internet could still contact the server as expected, i.e. ssh, ping and browse to it, I guess it’s not my network setup that is causing this issue.
As on the firewall (router) the interfaces for sn1 and sn2 are configured only with „pass any IPv4“ rules, I guess it’s not the firewall that is causing this issue.
As more than 24 hours later the problem still persisted for the affected client, I guess it’s not the server’s /etc/pf.conf that is causing this issue.

root@server:~ # less /etc/pf.conf

if = "{ bge0 }"
table <bruteforce> persist
icmp_types = "echoreq"
open_tcp = "{ 22, 25, 80, 110, 143, 443 }"
open_udp = "{ 22, 80 }"
jails = "{ 10.2.1.1 }"
ip = 10.1.2.2
jhzo_tcp_udp = "{ 80, 443 }"
hzo = 10.2.1.1

set block-policy drop
set skip on lo0
set loginterface bge0
set require-order yes
set fingerprints "/etc/pf.os"
set ruleset-optimization basic

scrub in all fragment reassemble random-id

rdr on $if proto {tcp udp} from any to $if port $jhzo_tcp_udp ->$hzo

nat on $if proto {tcp udp icmp} from $jails to any -> $ip

block quick from <bruteforce>
block log all
block return

antispoof quick for $if

pass in on $if proto tcp from any to any port $open_tcp flags S/SA keep state (max-src-conn 100, max-src-conn-rate 15/5, overload <bruteforce> flush global)
pass in on $if proto udp from any to any port $open_udp keep state
pass out log on $if inet proto tcp from $hzo to any port 25 flags S/SA synproxy state
pass out quick all keep state
pass in on $if inet proto icmp all icmp-type $icmp_types keep state
pass in on $if inet proto udp from any to any port 33433 >< 33626 keep state

/etc/pf.conf (END)

I finally rebooted the server and noted that after this the client could access it again. So I think the problem is somehow caused by the server, but I really don’t see any possible cause for this kind of interruption.
Do you? Any input is very welcome.

Thank you in advance and I’m happy to post any further information as requested.

 

[SirDice]

Check if the client doesn't end up in the bruteforce table. That would certainly block its access.

 

[stoe]

Thank you SirDice, you were correct... I'll have to find out now how a local client can match max-src-conn-rate 15/5 and end in this table.

I've now added the macro lan = "10.1.1.0/24" as well as a new line 34 of

pass in on $if proto tcp from $lan to any port $open_tcp flags S/SA keep state

to /etc/pf.conf and changed its now line 35 to

pass in on $if proto tcp from !$lan to any port $open_tcp flags S/SA keep state (max-src-conn 100, max-src-conn-rate 15/5, overload <bruteforce> flush global)

Thank you again.

 

[Lamia]

Immediately I saw a bruteforce table in the pf.conf, I thought it could be a culprit. Sshguard does the same.

If you are using snort/bro too, do add an exception to not flag that block of IP addresses.