PF blacklistd - manage /var/db/blacklistd.db

I'm wondering if it is possible to manage blacklistd.db

Code:
file /var/db/blacklistd.db                                                                  
/var/db/blacklistd.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)

I want to have option to remove entry from this file

and I'm curious why blacklistctl doesyn't have this option

any Idea?



PS. another tool shows sth like that:

Code:
db_dump185-6.2 /var/db/blacklistd.db                                  
format=bytevalue                                                                                                      
type=hash
h_ffactor=32                                              
db_lorder=1234                                                                                                        
db_pagesize=4096                                          
HEADER=END                                                
10020016530c0feb0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000020000000160000000600000002000000ffffffff03000000626c61636b6c697374640000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ffffffff80510100              
04000000000000002ddb7c59000000004f4b0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
100200164e42ad450000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000020000000160000000600000002000000ffffffff03000000626c61636b6c697374640000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ffffffff80510100
 
Wozzeck.Live Thanks for You reply,

i know how to remove/check blacklisted IP from firewalls, that's not my problem. But as You mention, that removal tool from blacklistd doesn't exists, and I'm curious if any of us (users of BSD) resolved that problem and have any idea how to do that.
 
Last edited by a moderator:
[FONT=Tahoma]Wozzeck.Live ... and another thing:[/FONT]

Wozzeck.Live said:
If you get tired of unbanning several known IPs you should add before the anchor a "quick" rule to override the anchor rule
(Actually I don't know if blacklistd creates a "quick" rule or a normal priority rule which will follow the global policy "last matching rule policy" of PF).

[FONT=Tahoma]So one should insert the anchors at the right point in its PF config file, and so for example just before the anchor we can insert the
following rule :
Code:[/FONT]
pass in quick from <TrustedSSH> to any port ssh

for blackllistd You can do this in that way /etc/blacklistd.conf

Code:
# Blacklist rule                     
# adr/mask:port type    proto   owner           name    nfail   disable   
[local]                               
wan0:ssh                stream  *       *               *       3       24h

# adr/mask:port type    proto   owner           name    nfail   disable
[remote]                             
_IP_ADDRESS_HERE_:ssh        *       *       *       *       *       *

so blacklistd never do banning _IP_ADDRESS_HERE_
 
Last edited by a moderator:
I'm wondering if it is possible to manage blacklistd.db

Code:
file /var/db/blacklistd.db                                                               
/var/db/blacklistd.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)

As there isn't any solution as of yet, here is a pythonic way to decode the hex blurbs.

And with a database opened with [FONT=Courier New]mode='w'[/FONT] you may also del [FONT=Courier New]db[key][/FONT] if it needs to go. Probably better stop blacklistd before modifying its database.

Code:
#!/usr/local/bin/python2
# (c) <nlsp@me.com> January 10, 2019
import socket
import struct
import datetime
import operator
# https://pypi.org/project/bsddb185
import bsddb185
def db_open(mode='r', path='/var/db/blacklistd.db'):
    return bsddb185.hashopen(path, mode)

class BlacklistDbConf(tuple):
    """named-tuple style blacklistdb key object"""
    __slots__ = ()
    _sockaddr_union_peek = struct.Struct('!BB')
    _inet4_sockaddr = struct.Struct('!BBH4s8s')
    _inet6_sockaddr = struct.Struct('!BBHH16sH')
    _db_conf_fields = struct.Struct('=lLLLll128sll')
    _proto_by_num = {
        socket.getprotobyname('tcp'): 'tcp',
        socket.getprotobyname('udp'): 'udp',
    }
    _family_by_num = {
        socket.AF_INET: 'inet',
        socket.AF_INET6: 'inet6',
    }
    def __new__(cls, arg):
        if isinstance(arg, tuple):
            return super(BlacklistDbConf, cls).__new__(cls, arg)
        return cls._unpack(arg)
    @classmethod
    def _unpack(cls, s):
        sockaddr_len, sockaddr_family = cls._sockaddr_union_peek.unpack(s[:2])
        # IPv6 never tested here, but include the fields anyway
        sin6_flowinfo = sin6_scopeid = None
        if sockaddr_family == socket.AF_INET:
            ( # IPv4 fields
              sin_len, sin_family, sin_port, sin_addr, sin_zero
            ) = cls._inet4_sockaddr.unpack(s[:sockaddr_len])
        elif sockaddr_family == socket.AF_INET6:
            ( # IPv6 fields
              sin6_len, sin6_family, sin_port, sin6_flowinfo, sin_addr, sin6_scopeid
            ) = cls._inet6_sockaddr.unpack(s[:sockaddr_len])
        else:
            raise ValueError, "unsupported address family number %d" % sockaddr_family
        addr = socket.inet_ntop(sockaddr_family, sin_addr)
        ( lmask, port, proto, family, uid, nfail, name, rmask, duration
        ) = cls._db_conf_fields.unpack(s[128:])
        proto = cls._proto_by_num.get(proto, proto)
        family = cls._family_by_num.get(family, family)
        return cls((
            addr, sin_port, sin6_flowinfo, sin6_scopeid,
            lmask, port, proto, family, uid, nfail, name.rstrip('\0'),
            rmask, datetime.timedelta(seconds=duration)
        ))
    ( # match _unpack return field order
      addr, sin_port, sin6_flowinfo, sin6_scopeid,
      lmask, port, proto, family, uid, nfail, name, rmask, duration
    ) = map(lambda n: property(
      lambda self, item=operator.itemgetter(n): item(self)
    ), range(13))
    def pack(self):
        if self.family == 'inet':
            packer = self._inet4_sockaddr
            ss_family = socket.AF_INET
            sin_addr = socket.inet_pton(ss_family, self.addr)
            args = (
                packer.size, ss_family, self.sin_port, sin_addr, ''
            )
        elif self.family == 'inet6':
            packer = self._inet6_sockaddr
            ss_family = socket.AF_INET6
            sin6_addr = socket.inet_pton(ss_family, self.addr)
            args = (
                packer.size, ss_family, self.sin_port, self.sin6_flowinfo, sin6_addr, self.sin6_scopeid
            )
        else:
            raise ValueError, "Address family %s not supported" % self.family
        addr = packer.pack(*args)
        pad = '\0' * (128 - packer.size)
        conf = self._db_conf_fields.pack(
            self.lmask, self.port, socket.getprotobyname(self.proto), ss_family,
            self.uid, self.nfail, self.name, self.rmask, int(self.duration.total_seconds())
        )
        return ''.join((addr, pad, conf))
    def __repr__(self):
        mask = self.rmask
        if mask == -1: mask = self.lmask
        return "<%s: %s %s %s/%d (port %d, nfail %d, name '%s', duration %s)>" % (
            self.__class__.__name__,
            self.family, self.proto, self.addr, mask,
            self.sin_port, self.nfail, self.name, self.duration
        )

class BlacklistDbEntry(tuple):
    """named-tuple style blacklistdb value object"""
    __slots__ = ()
    _db_entry_fields = struct.Struct('@QL64s')
    _epoch = datetime.datetime.fromtimestamp(0)
    def __new__(cls, arg):
        if isinstance(arg, tuple):
            return super(BlacklistDbEntry, cls).__new__(cls, arg)
        return cls._unpack(arg)
    @classmethod
    def _unpack(cls, s):
        entry_count, entry_last, entry_id = cls._db_entry_fields.unpack(s)
        return cls((
            entry_count,
            datetime.datetime.fromtimestamp(entry_last),
            entry_id.rstrip('\0')
        ))
    ( count, last, id ) = map(lambda n: property(
      lambda self, item=operator.itemgetter(n): item(self)
    ), range(3))
    def pack(self):
        return self._db_entry_fields.pack(
            self.count, int((self.last - self._epoch).total_seconds()), self.id
        )
    def __repr__(self):
        id = len(self.id) and ", id '%s'" % self.id or ''
        return "<%s: %s (count %d%s)>" % (
            self.__class__.__name__,
            self.last, self.count, id
        )

class BlacklistEntry(object):
    """demo class to actually sort and present some data"""
    __slots__ = ('conf', 'entry', '_time')
    def __init__(self, conf, entry, time=None):
        self.conf = conf
        self.entry = entry
        self._time = time
    @property
    def time(self):
        if self._time is None:
            return datetime.datetime.now()
        return self._time
    @property
    def remain(self):
        return self.entry.last - self.time + self.conf.duration
    def __str__(self):
        blocked = bool(self.entry.id) and "blocked" or "%d/%d" % (
            self.entry.count, self.conf.nfail
        )
        return "%s %s/%s port %d (%s) (%s remaining)" % (
            self.conf.proto, self.conf.addr, self.conf.lmask,
            self.conf.port, blocked, self.remain,
        )
"""(fun fact: line number 185 here)"""
if __name__ == '__main__':
    db = db_open('r')
    now = datetime.datetime.now().replace(microsecond=0)
    items = []
    debug = 4
    if debug > 1:
        import pprint
    for k in db.keys():
        try:
            v = db[k]
        except KeyError:
            continue
        if debug > 2: print k.encode('hex')
        if debug > 2: print v.encode('hex')
        conf = BlacklistDbConf(k)
        if debug > 3:
            repack = conf.pack()
            assert repack == k, "packed value does not match binary:\n%s\n" % repack.encode('hex')
        entry = BlacklistDbEntry(v)
        if debug > 3:
            repack = entry.pack()
            assert repack == v, "packed value does not match binary:\n%s\n" % repack.encode(hex)
        entry = BlacklistDbEntry(v)
        if debug > 1: pprint.pprint(map(tuple, (conf, entry)))
        if debug > 1: print ""
        if debug > 0: print(conf)
        if debug > 0: print(entry)
        if debug > 0: print ""
        items.append(BlacklistEntry(conf, entry, now))
    items.sort(key=operator.attrgetter('remain'), reverse=True)
    for i in items:
        print i
    db.close()
 
Oh my, there's a bug in the BlacklistDbConf.pack() inet6 case. Should be (corrected now):

Code:
        elif self.family == 'inet6':
            packer = self._inet6_sockaddr
            ss_family = socket.AF_INET6
            sin6_addr = socket.inet_pton(ss_family, self.addr)

Obviously. And the forum squeezed away all blank lines, making the 185 linenumber joke moot.
Shame on me for a first post :-//)
 
Back
Top