FreeBSD 12 log format (fail2ban not matching)

[usdmatt]

Anyone else noticed a different log format in FreeBSD 12?
I have two servers running 12, one which seems to show the old format, and one using a new format, and I can't figure out why. It's stopping any fail2ban filters from correctly matching.

Server 1

# uname -a
FreeBSD xxx.yyy.zzz 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64
# tail /var/log/messages
Dec 19 15:11:49 xxx pure-ftpd[29784]: (?@MY-IP) [WARNING] Authentication failed for user [matt]

Server 2

# uname -a
FreeBSD aaa.bbb.ccc 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64
# tail /var/log/messages
Dec 19 15:01:59 ftp 1 2018-12-19T15:01:59.756086+00:00 aaa.bbb.ccc pure-ftpd 69456 - - (?@MY-IP) [WARNING] Authentication failed for user [matt]

I came across the following review which seems to discuss a similar format, but can't see why it's different between two servers running an identical OS. https://reviews.freebsd.org/D14926

 

[obsigna]

After looking into my 12.0 and 11.2 logs, and then comparing the obvious differences in your logs, I come to the conclusion, that pure-ftpd is responsible for the different messages. Are we talking about the same versions of ftp/pure-ftpd? Or perhaps (I know of pure-ftp only that it exists), is there a setting in the configuration file where you can specify the log format?

 

[usdmatt]

I thought the same thing but then I seem to have pkg entries in the same format. This is in /var/log/messages so I didn't think the source daemon had much say over the log format.

Dec 17 15:27:46 ftp 1 2018-12-17T15:27:46.576942+00:00 aaa.bbb.ccc pkg-static 75241 - - pkg-1.10.5_5 installed

The only difference with pure-ftpd is that I installed from ports on this machine in order to enable MySQL support.

If anyone has a similar issue, I managed to get fail2ban working after about an hour of battling regexes by adding a /usr/local/etc/fail2ban/filter.d/common.local containing the following.

[DEFAULT]

__rfc5424_timestamp = [\d\-]+T[\d:]+\.\d+\+[\d:]+
__rfc5424_facility = \S+
__rfc5424_version = \d+
__rfc5424_prefix = %(__rfc5424_facility)s\s%(__rfc5424_version)s\s%(__rfc5424_timestamp)s
__rfc5424_pid_extra = \d+\s\-\s\-

__prefix_line = %(__date_ambit)s?\s*(?:%(__rfc5424_prefix)s\s)?(?:%(__hostname)s\s)?(?:%(_daemon)s\s)?(?:%(__rfc5424_pid_extra)s)?\s+

 

[woodsb02]

I was just discussing this issue with usdmatt on the freebsd-hackers mailing list, and we established it was an issue where 1 more reboot was required after the upgrade to FreeBSD 12.

The problem being that the old syslogd process was still running, and expecting log messages from /var/run/log[priv] to be in the old RFC3164 format, whilst new processes were running (such as ssh) and logging to syslog(3) which was writing to the log socket in the new RFC5424 format. A restart ensures all processes are using the new libc syslog(3) function and the new syslogd(8) daemon is running.