PF Over 390 000 IP addresses from a DDoS attack in a text file

[Aknot]

I have collected over 390 000 IP addresses from a DDoS attack in a text file.
It's collected from /var/log/httpd-access.log with awk, like this:

awk '$11 == "404" && index($9, "/url-being-attacked") {print $1}' /var/log/httpd-access.log | sort -n >> /usr/local/etc/pf.ip.ddos.list

Then I'm removing all the duplicates ant sorting the IP addresses:

sort pf.ip.ddos.list | uniq > pf.ddos.list

Then the list is loaded to a pf table:

### DDoS Table
table <ddos> persist file "/usr/local/etc/pf.ddos.list"
block in quick from <ddos> to any

The file is big now, and contains roughly 390 000 IP addresses.
There are roughly 10 000 hits every 5 minutes.
FreeBSD handles it fine, but php-fpm is spiking the CPU cores too 100%

Many of the IP addresses is coming from the same "infected" net, like this:
(the host IP isn't shown, the zeros represent different host IPs)

...
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
103.111.225.0
…

The question: Is there any script out there, that can check the net IP, and make a CDIRs to a file instead?

103.111.225.0/32

It would be so much easier to maintain.
Or are there any limitations to pf and the table sizes?

I have made the following changes:

sysctl net.pf.request_maxcount=1500000
set limit table-entries 1500000

Is btw sysctl net.pf.request_maxcount=1500000 lost upon reboot?

Thanks,

 

[SirDice]

It's a network address, not a host IP.

% Information related to '103.111.225.0/24AS137526'

route:          103.111.225.0/24
origin:         AS137526
descr:          Plusnet Inc
                Goribullah R/A, Khulshi 4225
                Chittagong
mnt-by:         MAINT-PLUSNETINC-BD
last-modified:  2019-11-20T11:04:15Z
source:         APNIC

 

[Aknot]

It's a network address, not a host IP.

Bad explained, I'm updating the post. The IP addresses is just an excerpt from the log; I'm hiding the last host IP number with a zero.

 

[grandpa]

...

The question: Is there any script out there, that can check the net IP, and make a CDIRs to a file instead?
...

I wouldn't recommend turning all ips to CIDRs since it's possible that you will block non-attackers as well. I have malicious stuff coming in from datacenters hosting DigitalOcean, Amazon, Microsoft or Oracle or whatever and in the same CIDRs there do exist services that are legit.

With that in mind here's an idea - Script an RDAP lookup with a polite throttling and grep the CIDR. It will take a long time to do this so it's not ideal for realtime protection. You could do the same or combine it with a whois lookup.

If you want to be granular about blocking either ip or CIDR could a count of how many ips you have from the reported CIDR and use that a a trigger. Or filter based on net owner.

/grandpa

 

[atax1a]

pf tables get built into an efficient structure. you can exploit that structure from other programs by testing the return code of pfctl -q -t ddos -T test 203.0.113.1 which will return 0 if the address is in the table, and 1 if not.

 

[JLAIP]

I wouldn't recommend turning all ips to CIDRs since it's possible that you will block non-attackers as well. I have malicious stuff coming in from datacenters hosting DigitalOcean, Amazon, Microsoft or Oracle or whatever and in the same CIDRs there do exist services that are legit
/grandpa

DigitalOcean, Amazon, Microsoft, Oracle....troublemakers, all. Feh.

In general, I'd agree that Aknot's brute force approach may block some legitimate visitors (I routinely block [static] IP ranges from the hosts you cited based on their....well, let's just say their questionable activities). But, on the other hand, when you have, say, 30+ miscreants within a range of 254 (who instantly return from an adjacent IP within seconds of being blocked!) up to no good and many (most?) of these IPs're static, I'd block 'em, too. And do!

As mentioned in a coupla my recent posts, bad actors've become so ingrained into the system and potentially disruptive, often outnumbering legitimate traffic from the same IP ranges, that some of us believe it's time to fight brute force [attacks] with brute force [solutions] to protect our farms from the feral animals.

Yes, for those with the skills/time to repeatedly craft detailed scripts to fine-tune targeted protective measures, only to have to repeat the process as the arms race progresses, brute force measures may appear silly. But they tend to work. At least that's been my experience.....and we've not seen any reduction in legitimate traffic on our servers with the application of /24 (and, in extreme cases, /16) CIDR (subnet) blocks.