Adding DNS blocklist to apache

Blocking large number of offensive hosts is possible by editing .htaccess, BUT this forces a restart each time a change is done and takes cpu from apache.
A better and more scalable method is use of a dns blocklist
I have been running libapache2-mod-defensible_1.4-3.2_amd64.deb
several years with good effects. All needed to adjust the list of blocked ip is to add to the
zonefile and "rndc reload" the promary dns server. All features of DNS is available including
replication via slave servers and caching in local resolving servers.

The above "mod-defensible" is not ported to freebsd ports. I am not sure i can do it, but
a skilled maintainer could probably do this quite easy.

Any takers ? I have at least one server running apache 2.4 to test/verify on.
 
you can implement a rewrite map program which you feed "%{REMOTE_HOST}" and the look it up
on the bl dns
if it resolves send 403 url, otherwise send -
 
The simplicity with defensible is that 2 lines are present
in /etc/apache2/mods-enabled/defensible.conf

"
DnsblUse On
DnsblServers webaccess.manet.nu
"
 
in vhost definition
RewriteMap rbl "prg:/some/path/z.sh"
z.sh
Code:
#!/bin/sh
IFS="."
while read a
do
set -- $a
host $4.$3.$2.$1.zen.spamhaus.org >/dev/null 2>&1 && echo "403.html" || echo "NULL"
done
in .htaccess
RewriteCond %{REQUEST_URI} .
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^ ${rbl:%{REMOTE_HOST}|%{REQUEST_URI}} [L,QSA]
 
you should probably used a script written in a language that support gethostbyname so you don't need to run an external program for every request
 
better version
.htaccess
Code:
RewriteCond ${rbl:%{REMOTE_HOST}|NOT_FOUND} !NOT_FOUND
RewriteRule ^ /403.php?ip=%{REMOTE_HOST} [END]
map program (php)
PHP:
#!/usr/local/bin/php
<?php
$maplog = fopen("/tmp/maplog.txt","w");
while($ip = trim(fgets(STDIN))) {
$host = implode(".",array_reverse(explode(".",$ip))) . ".zen.spamhaus.org";
$resolv = gethostbyname($host);
if($resolv == $host) {
 echo "NULL\n";
 fputs($maplog,"NULL\n");
 } else {
 fputs($maplog,"403.html?ip=$ip\n");
  echo "403.html?ip=$ip\n";
  }
}
 
Thanks for the examples. Back to my original question :
Would porting defensible be hard work ? It does work
very good, has no problems and is available on most linux distos, why
should freebsd be missing this ?
 
probably lack of interest/men power
i speculate it won't be technically hard but someone still needs to take his time to create a port and then maintain it
 
mod_defensible is not listed for Apache 2.4. A quick google search told me that Apache 2.2 used to support it, but it was not an official module - Linux distros did cobble it together, but Apache project chose to focus on HTTP/HTTPS specifically, rather than include DNS. FreeBSD ports infrastructure maintainers apparently decided to only include stuff that's officially supported by Apache (This is just an educated guess on my part). For DNS blacklisting, OP might want to read this.
 
FreeBSD ports infrastructure maintainers apparently decided to only include stuff that's officially supported by Apache (This is just an educated guess on my part).
Only the modules you can enable/disable in the www/apache24 port itself. But that's a maintainer decision. There are a couple of other Apache modules available as separate ports, there's nothing stopping anyone from submitting them. Nothing is being "sanctioned" here, if you create a proper port it will get accepted, it's as simple as that. You just have to find someone willing and able to create a port for it and keep it maintained.

Whether or not you should do this filtering on the Apache host itself is certainly debatable. I would probably use a reverse proxy for this, do the initial filtering there and let Apache do what it does best, serve pages and nothing else.
 
Two comments :
astyle: the apache link warns for using dns info in config files. Good advice.
SirDice: refering to a reverse proxy does not help using a dns-based blocklist
The beauty with a dns based blocklist ( just as have been done in mail servers for
many years) is that updating the blocklist may be done without disrupting service.
Using dns instead of files gives a stable infrastructure with a redundant server
network enabling sharing of this list to an unlimited set of users.
Blocking "bad requests" might actually reduce load as apache don't have to
make complex decisions but instead returns a fixed small text message. The
cost of a query/response is indeed very small. Searching a large mass if
ip addresses in .htaccess ( the remaining method) is much more resource
intensive, more as the list grows.

Summary a dns-based blocklist scales very well, and workload is independent
of list size. The list might be updated at any time without disturbing normal
operations of apache. Same goes for the dns server part, updating and maintaning a list ( aka dns zone) will not disturb operations of dns servers.

the module defensible has been used > 5 years by me, but i have been
forced to use various linux distros. To use FreeBSD as host would be much
easier and reliable for me, thus my interest in this issue.
 
peter-h : it may help to review the TCP/IP stack and connect the dots: Where BIND works, where the firewall (PF/IPFW) is, and where Apache accepts the HTTP(S) requests. You basically shouldn't ask Apache to take on the duties of a firewall or name resolution services. Once you kind of nail down what hoops a connection request goes through (and in what order) before reaching the web server, an easy solution for DNS-based blocking will become obvious. 'Best Practice' recommendations will also start making more sense.
 
peter-h : it may help to review the TCP/IP stack and connect the dots: Where BIND works, where the firewall (PF/IPFW) is, and where Apache accepts the HTTP(S) requests. You basically shouldn't ask Apache to take on the duties of a firewall or name resolution services. Once you kind of nail down what hoops a connection request goes through (and in what order) before reaching the web server, an easy solution for DNS-based blocking will become obvious. 'Best Practice' recommendations will also start making more sense.
I'm perfectly aware of the ISO / TCPIP layered stack thank you.
Thats why i would not suggest pf or similar use filters that is not core resident.
tcpwrapper (tcpd) is also out since it will be a performance hog ( although it could make sense to have tcpwrapper have a possibility to use dns).

On the contrary i want apache just like sendmail ( or close to all other mta ) to
use an external dns based blocklist. The box(es) in case already has proper
shielding using netfilter (as linux).
If you read my first post you see that i want apache to use the resolver library
to act as a dns client. Server side works very well with bind et.al.

Best practice as referred to in the case of MTA is to have the MTA decide what
to accept. Noone uses PF/IPF to choose from who to accept mail. Same
for apache.
 
Please note that i'm not here to start a flamewar. I wanted to probe the
interest for porting this "defensible" to the freebsd port tree. If
noone is interested i'll just continue to use linux.
I guess i will drop my presence here ( and wish you all good luck, happy x.mas
and a general success. Freebsd rocks !)

peter h
 
I just wanted to point out that there's usually more than one way to solve a problem, but no need to reinvent the wheel.
--
I do think that the 'mod-defensible' reinvents the wheel here. Sometimes, it helps to write a helper script that calls an API in just the right spot, and sometimes, it may be cleaner to step back and re-think the whole approach to the problem.
 
I just wanted to point out that there's usually more than one way to solve a problem, but no need to reinvent the wheel.
--
I do think that the 'mod-defensible' reinvents the wheel here. Sometimes, it helps to write a helper script that calls an API in just the right spot, and sometimes, it may be cleaner to step back and re-think the whole approach to the problem.
Yes, i have done that. defenseble is the way, no overhead, no scripts involved just like DNS-BL in (any)MTA. In addition it's scalable and usable in many apaches if one has, and it allows updating the list "on the fly" without disturbing service.
 
Many ports exist because someone though it was a useful application. The mantra is basically "If you build it, they will come." Not lets see who's interested first, then ask someone else to do the work. The question is, how badly do you want it? Do you want it enough to invest your time and energy into trying to port it? Yes, you will run into problems initially, but when you've done some legwork yourself I found it quite easy to find help on specific issues I was running into.

FreeBSD Porter's Handbook
 
We are moving, mod_defensible is running on 2 freebsd machines of different age. Work on a port is started.
 
A part time report,
i am running a 1574453583 hosts blocklist with mod_defensible and have no
scaling problems. Not possible with any of the alternatives !
 
Back
Top