Solved ICMP commands fails until Pf is reloaded

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

On starting my PC, I could not get the icmp commands (ping, etc) to work. Needless to say services like email don't work but not web servers, which surprisingly work. Webpages are accessible. I always have to run "service pf reload" but email server and icmp commands would work.

Below is my pf rules

Code:
..........
..........
..........
...........
# Allowed ips for ping
ping_outbound_ips  = "{ 192.168.1.2 }"
traceroute_outbound_ips  = "{ 192.168.1.2 }"

# Fine-tuning pf rules using freebsd forum 50364
# Follow RFC1918 and don't route to non-routable IPs
# http://www.iana.org/assignments/ipv4-address-space
# http://rfc.net/rfc1918.html
nonroute= "{ 0.0.0.0/8, 20.20.20.0/24, 127.0.0.0/8, 169.254.0.0/16, 224.0.0.0/3, 255.255.255.255 }"


# Set allowed ICMP types
icmp_types = "{ 0, 3, 4, 8, 11, 12 }"

### Tables ###
table <sshguard> persist
table <bruteforce> persist

# [options]
####disable filtering(nat, redirect) on loopback lo0
set skip on lo0
set skip on lo1
set skip on lo2
# Don't send rejections. Just drop.
set block-policy drop
set limit               { states 40000, frags 20000, src-nodes 20000 }
set loginterface    $ext_if
set optimization    aggressive
set timeout        { interval 10, frag 30, adaptive.start 12000, adaptive.end 24000 }
set debug loud
set fingerprints "/etc/pf.os"
set state-policy if-bound


# [normalizaiton]
# all incoming traffic on external interface is normalized and fragmented packets are reassembled.
scrub in on $ext_if all fragment reassemble


# NAT rules
nat pass on $ext_if proto tcp from $netjail_1 to any -> $ip_pub1
nat pass on $ext_if proto udp from $netjail_1 to any -> $ip_pub1



# RDR rules
rdr pass on $ext_if proto tcp from any to $ip_pub2_4mail port $jail2_port -> $netjail_2


# [translation]



# [filtering]
block in log all
# Enable antispoofing on the external interface
antispoof for $ext_if inet
antispoof for $ext_if inet6
pass out all
#next line make it possible to use & access ssh at prt 3456 and...
#the next lines are specifically for the ssh in order to be a little more tight fisted for ssh
pass log quick proto tcp from any to any port $ssh_port \
        flags S/SA keep state \
        (max-src-conn 50, max-src-conn-rate 10/5, \
        overload <bruteforce> flush global)

# block packets that fail a reverse path check. we look up the routing
# table, check to make sure that the outbound is the same as the source
# it came in on. if not, it is probably source address spoofed.
block in from urpf-failed to any


block quick from <bruteforce>

pass in log on $ext_if proto {tcp, udp} from any to $ext_if port $tcp_pass \
     keep state (max-src-conn 500, max-src-conn-rate 15/5, \
     overload <bruteforce> flush global)
# block all IPs from  sshguard-pf blocklist without any further evaluation
block drop in log quick on $ext_if inet from <sshguard> to any


#block nmap and fingerprint probing
block in quick on $ext_if proto tcp flags FUP/WEUAPRSF
block in quick on $ext_if proto tcp flags WEUAPRSF/WEUAPRSF
block in quick on $ext_if proto tcp flags SRAFU/WEUAPRSF
block in quick on $ext_if proto tcp flags /WEUAPRSF
block in quick on $ext_if proto tcp flags SR/SR
block in quick on $ext_if proto tcp flags SF/SF
block in quick on $ext_if proto tcp from any to any flags FUP/FUP
block in log quick on $ext_if from any os "NMAP" to any label ExtNMAPScan


# silently drop broadcasts cable modem noise
block in quick on $ext_if from any to 255.255.255.255

# block anything coming from source we have no back routes for
block in from no-route to any

# Drop packets to non-routable addresses directly
block drop out quick on $ext_if from any to $nonroute

# Block and log outgoing packets that don't have our address as source,
# they are either spoofed or something is misconfigured NAT disabled,
# (for instance), we want to be nice and don't send out garbage.
block out log quick on $ext_if from !$ext_if to any


###allow filtering(nat, redirect) on the loopbacks lo0 lo1 lo2 lo3 lo4
pass in log on lo0 proto tcp from any to any flags S/SA keep state
pass out log on lo0 proto tcp from any to any flags S/SA keep state
pass in log on lo1 proto tcp from any to any flags S/SA keep state
pass out log on lo1 proto tcp from any to any flags S/SA keep state



# Allow traffic out of open ports
pass out on $ext_if proto tcp to any port $tcp_pass keep state
pass out on $ext_if proto udp to any port $udp_pass keep state


####freebsd forum 9146 suggests hosts should allow icmp in - some CDNs ping hosts to determine nearest mirrors
pass in on $ext_if inet proto icmp to ($ext_if) icmp-type $icmp_types keep state
#allow ping out of the jail(s)
pass out on $ext_if inet proto icmp from $ping_outbound_ips to ($ext_if) icmp-type $icmp_types keep state

#allow Unix traceroute & allow out the default range for traceroute
pass out on $ext_if inet proto udp from $traceroute_outbound_ips to any port 33433 >< 33626 keep state


And some point, I was thinking that the fingerprint messages on my QEMU terminal are the cause of the problem.
qemu.png



The QEMU terminal messages and like the messages in my TCP dump, though they were more of IPv6. The below is my TCPdump log:

Code:
# tcpdump -i pflog0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 262144 bytes
00:36:10.168182 IP 91.13.....77.80 > 38......74.50392: Flags [S.], seq 3537156732, ack 620526304, win 17520, length 0
00:36:13.342000 IP6 2a02..........:1771 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::1, length 32



My netstat log is this:

Code:
# netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            ams1-pod6-vc6-v1-2 UGS      et0
35......0/24    link#1             U        et0
majk.tk.comn link#1             UHS         lo0
85......0/24     link#1             U        et0
localhost          link#1             UHS         lo0
localhost          link#2             UH          lo0
127.0.1.1          link#3             UH          lo1
192.168.1.2        link#3             UH          lo1

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
default            ams1-pod6-vc6-v1-1 UGS      et0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
2a....::/64   link#1             U        et0
hulol           link#1             UHS         lo0
fe80::/10          localhost          UGRS        lo0



I am thinking these packets are flooding my server and preventing it from successufully running the ICMP commands. There are a few times when I restart my PC and the icmp commands would work without necessarily running "service pf reload" first. Kindly guide me on how to get the icmp commands to work always. The PC, even after reloading the pf rules, overtime stops running the ICMP commands again and I would need 'manually' reload the pf rules again. My email server on checking its maillog or on running "postqueue -p" would show errors like "host unreachable or unavailable". In addition, dovecot stops working with a fatal error in maillog. Emails sent to our server are also undelivered until we reload the pf rules and the emails are resent. Perhaps, I should disable IPv6 to fix it. My rc.conf is fine; a few tricks I tried in it included - set/unset defaultrouter; changed from dhcp to static IP, and vice versa, for the interface ip address, etc but no luck.

Thanks in advance.
 
Last edited:

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 10,890
Messages: 36,805

I am thinking these packets are flooding my server and preventing it from successufully running the ICMP commands.
Look at the timestamps of the tcpdump(1) output. There are, at most, a couple of packets per second. That's nowhere near even the most conservative definition of a "flood". Flooding typically requires hundreds, thousands or millions of packets per second.
 
OP
Lamia

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

Look at the timestamps of the tcpdump(1) output. There are, at most, a couple of packets per second. That's nowhere near even the most conservative definition of a "flood". Flooding typically requires hundreds, thousands or millions of packets per second.
Hello SirDIce, thanks for your comment; I am learning from everyone of you, the experts. Do you know how I can resolve the problem - icmp commands not working & a few services down after (re)starting my PC until I run 'service pf reload'? And reloading pf fails to fix it at some times. At those times, I have no other choice than to restart the PC.
 

Trihexagonal

Daemon

Reaction score: 1,675
Messages: 2,249

Could you please post the results of # pfctl -s rules so we could see how your outbound ICMP rule is being handled?
 
OP
Lamia

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

Could you please post the results of # pfctl -s rules so we could see how your outbound ICMP rule is being handled?


This <textarea> would not accept all the lines so I ended up splitting them.

Thanks Trihexagonal.

Code:
pass in log on et0 inet6 proto udp from any to fe80::1 port = https keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet6 proto udp from any to fe80::1 port = smtps keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet6 proto udp from any to fe80::1 port = submission keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet6 proto udp from any to 2a....: port = imap keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet6 proto udp from any to 2a....: port = https keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in on et0 inet proto icmp from any to (et0) icmp-type unreach keep state (if-bound)
pass in on et0 inet proto icmp from any to (et0) icmp-type squench keep state (if-bound)
pass in on et0 inet proto icmp from any to (et0) icmp-type echoreq keep state (if-bound)
pass in on et0 inet proto icmp from any to (et0) icmp-type timex keep state (if-bound)
pass in on et0 inet proto icmp from any to (et0) icmp-type paramprob keep state (if-bound)
 
Last edited:
OP
Lamia

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

Here is the first half/part of the log:
Code:
# pfctl -s rules
scrub in on et0 all fragment reassemble
block drop in log all
block drop in on ! et0 inet from 85......0/24 to any
block drop in inet from 85......0 to any
block drop in inet from 35.....0 to any
block drop in on ! et0 inet6 from 2a....:8::/64 to any
block drop in on et0 inet6 from fe80::1 to any
block drop in inet6 from 2a....: to any
pass out all flags S/SA keep state (if-bound)
pass log quick proto tcp from any to any port = 3456 flags S/SA keep state (source-track rule, max-src-conn 50, max-src-conn-rate 10/5, overload <bruteforce> flush global, if-bound, src.track 5)
block drop in from urpf-failed to any
block drop quick from <bruteforce> to any
pass in log on et0 inet proto tcp from any to 85....0 port = domain flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85.......0 port = http flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85......0 port = ntp flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85......0 port = imap flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85.....0 port = https flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85......0 port = smtps flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
pass in log on et0 inet proto tcp from any to 85......0 port = submission flags S/SA keep state (source-track rule, max-src-conn 500, max-src-conn-rate 15/5, overload <bruteforce> flush global, if-bound, src.track 5)
<bruteforce> flush global, if-bound, src.track 5).<bruteforce> flush global, if-bound, src.track 5).
pass in log on et0 inet proto udp from any to 85.......0 port = ntp keep state (source-
 
Last edited:

Trihexagonal

Daemon

Reaction score: 1,675
Messages: 2,249

I'm not a firewall expert and don't "pretend" to be so nobody takes exception, but isn't that looping it back to the interface?

Code:
pass out on et0 inet proto icmp from <__automatic_596b31d6_1> to (et0) icmp-type echorep keep state (if-bound)

I only ask because my outbound ICMP rule is different:

Code:
pass out on $ext_if proto { tcp, udp, icmp } from any [b]to any[/b] modulate state
 
OP
Lamia

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

I'm not a firewall expert and don't "pretend" to be so nobody takes exception, but isn't that looping it back to the interface?

Code:
pass out on et0 inet proto icmp from <__automatic_596b31d6_1> to (et0) icmp-type echorep keep state (if-bound)

I only ask because my outbound ICMP rule is different:

Code:
pass out on $ext_if proto { tcp, udp, icmp } from any to any modulate state
Thanks Trihexagonal for your input, but it did not fix it.
 
OP
Lamia

Lamia

Aspiring Daemon

Reaction score: 182
Messages: 697

I am going to close this thread. I only found a way around the problem by setting a cron job that periodically run pf reload.

I could speculate that the problem is as a result of a number of services (IDS etc) that run at startup on the host machine. Someone somewhere might be able to replicate it; and here is a quick fix to it.
 
Top