Solved [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssassin

Hi there,

I've set[]up Dovecote2 + Postfix + SpamAssassin on my FreeBSD 10 VPS and it works flawlessly. Though, I couldn't set[]up the Dovecot Antispam plugin to train SpamAssassin when I move mails to the Junk folder.

I installed the plugin from ports and followed the official documentation and the man page without any success.

This is my /usr/local/etc/dovecot/dovecot.conf:
Code:
auth_mechanisms = plain login digest-md5
disable_plaintext_auth = no
first_valid_gid = 6
first_valid_uid = 26
last_valid_gid = 6
last_valid_uid = 26
lmtp_save_to_detail_mailbox = yes
mail_gid = mail
mail_location = mdbox:/mail/%d/%u
mail_privileged_group = mail
mail_uid = mailnull
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location =
  mailbox Archive {
    #auto = create
    auto = no
    special_use = \Archive
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Flagged {
    #auto = create
    auto = no
    special_use = \Flagged
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  mailbox virtual/All {
    auto = no
    special_use = \All
  }
  prefix =
  separator = /
}
passdb {
  args = /mail/%d/dovecot_passwd
  driver = passwd-file
}
plugin {
  antispam_backend = MAILTRAIN
  antispam_mail_spam  = --spam
  antispam_mail_notspam  = --ham
  antispam_mail_sendmail = /mail/sa-learn-pipe.sh
  antispam_spam = Junk
  antispam_trash = Trash
  quota_rule = *:storage=8GB
  sieve = /mail/%d/%u/dovecot.sieve
  sieve_dir = /mail/%d/%u
  sieve_global_dir = /mail/sieve
  sieve_global_path = /mail/sieve/globalfilter.sieve
}
protocols = imap pop3 lmtp sieve
service auth {
  unix_listener auth-client {
    group = mail
    mode = 0660
    user = postfix
  }
  unix_listener auth-master {
    group = mail
    mode = 0600
    user = mailnull
  }
  user = root
}
service imap-login {
  inet_listener imap {
    address = *
    port = 143
  }
  inet_listener imaps {
    address = *
    port = 993
  }
  process_limit = 32
  process_min_avail = 1
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 0
  service_count = 1
  vsz_limit = 64 M
}
service managesieve {
  process_limit = 1024
}
service pop3-login {
  inet_listener pop3 {
    address = *
    port = 110
  }
  inet_listener pop3s {
    address = *
    port = 995
  }
  process_limit = 32
  process_min_avail = 1
}
ssl_cert = </etc/ssl/dovecot/cert.pem
ssl_key = </etc/ssl/dovecot/key.pem
userdb {
  args = /mail/%d/dovecot_passwd
  driver = passwd-file
}
verbose_proctitle = yes
protocol lmtp {
  mail_plugins = $mail_plugins sieve
  postmaster_address = postmaster@SomeDomain.com
  quota_full_tempfail = yes
}
protocol sieve {
  mail_max_userip_connections = 10
  managesieve_implementation_string = Dovecot Pigeonhole
  managesieve_logout_format = bytes=%i/%o
  managesieve_max_line_length = 65536
}
protocol imap {
  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
  mail_plugins = $mail_plugins quota imap_quota antispam
}
protocol pop3 {
  mail_plugins = $mail_plugins quota
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}
protocol lda {
  mail_plugins = $mail_plugins quota sieve
  postmaster_address = postmaster@SomeDomain.com
  sendmail_path = /usr/local/sbin/sendmail
}

And, this the script /mail/sa-learn-pipe.sh:
Code:
#!/bin/sh

echo /usr/local/bin/sa-learn $* /tmp/sendmail-msg-$$.txt
echo "$$-start ($*)" >> /tmp/sa-learn-pipe.log

#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt

/usr/local/bin/sa-learn $* /tmp/sendmail-msg-$$.txt

rm -f /tmp/sendmail-msg-$$.txt

echo "$$-end" >> /tmp/sa-learn-pipe.log

exit 0

I made it excecutable by all users using chmod a+x /mail/sa-learn-pipe.sh

The problem is there is no log in /var/log/maillog regarding the Antispam plugin, and the script won't get called when I moved any message in/out to/from Junk folder.

I'll appreciate any help.
 
Re: Dovecot2 Antispam MAILTRAIN backend with SpamAssassin

For anyone coming across this thread: There were no issue with the script and or configuration in the first place.

Since I didn't received any response here, I posted a similar thread on Dovecot mailing list. And thanks to Leander Schäfer, I was trying his configuration and I saw a bit of difference and things started working. Then, I reverted back to my original configuration (the one I posted here) and it automagically started working.

Probably, it was a stupid mistake by me.

Cheers!
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

Did you get antispam to log anything verbose?

I'm trying to get the same configured and I can't for the life of me get it to call the helper script.
I've also tried SPOOL2DIR and that does not work either.

The only time I've seen antispam log anything is if I don't configure any backends (it then logs an error).

It's frustrating!
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

Sorry to answer the topic very late.
Please post your configuration for us to take a look, first.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

Thanks for replying.

I gave up trying to get it working. And now I'm taking a different approach.
Using the maillog plugin I've written a script to parse the dovecot logs and do the mail training based on the logged operations.

So far it's working out quite well, but I'm still fine tuning it. It gives much flexibility thank the antispam plugin.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

MrFollies said:
Thanks for replying.

I gave up trying to get it working. And now I'm taking a different approach.
Using the maillog plugin I've written a script to parse the dovecot logs and do the mail training based on the logged operations.

So far it's working out quite well, but I'm still fine tuning it. It gives much flexibility thank the antispam plugin.

It might be quite interesting. As you may know the antispam plugin was an efforts like yours for the Dovecot version 1 and someone else forked it for version two. It would be nice if you share your solution. It may become an official Dovecot pulgin.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

NuLL3rr0r said:
It might be quite interesting. As you may know the antispam plugin was an efforts like yours for the Dovecot version 1 and someone else forked it for version two. It would be nice if you share your solution. It may become an official Dovecot pulgin.

I'll post it to the dovecot list and also here if you like, when I'm happy with it.

Out of curiosity, how is the antispam plugin working out for you?
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

That would be great.

I'm quite satisfied with it since it made SpamAssassin learn what should be considered as spam or ham. After using the plugin for a while now, I must admit more than 90% of what should be considered as spam will end-up in the Junk folder, where they belong. Without SpamAssassin and specially the plugin itself my daily life was a disaster due the high amount of Chinese / Russian spam that I used to received in my inbox.

So, I highly recommend it.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

90%?

I can understand going from no filtering to a 90% success rate would feel good. I'm having better success with my numbers close to 100%. I think I see a junk message in my INBOX about once a month.
But the problem for me is I was re-building my database from scratch every night and the IO was a problem for my ISP (Virtualised Server).

I leave SpamAssasin alone, but have a custom filtering DB based on bogospam in the last steps of mail processing.

So my new solution is a "Build the DB once and then add and remove messages" approach. I hope to keep the same success rate.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

I've never heard of bogospam. But I'm going to try it on my spare time.

And, about the VPS, I'll highly recommend RootBSD. I got my Omicron VPS almost 4 years ago and it's still running fine and never had an issue. They run FreeBSD on Xen and their IO and network performance is awesome. Also they provide 15% discount using coupon codes (you could find some through googling) + another 10% when you are paying annually or semi-annually.
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

NuLL3rr0r said:
I've never heard of bogospam. But I'm going to try it on my spare time.

And, about the VPS, I'll highly recommend RootBSD. I got my Omicron VPS almost 4 years ago and it's still running fine and never had an issue. They run FreeBSD on Xen and their IO and network performance is awesome. Also they provide 15% discount using coupon codes (you could find some through googling) + another 10% when you are paying annually or semi-annually.

I think it's actually called bogofilter. Here it is : http://bogofilter.sourceforge.net/
It's just a very simple implementation of stats based filtering. There's just enough options to tweak to get a nice result.
On FreeBSD, I find the sqlite backend to be the most efficient.

I try and deny as much as possible at the MTA. So still quite a lot gets through. Bogofilter cleans up the rest nicely when it has a decent body of example messages.
Currently I'm building the DB with 100days history of spam and non-spam and hitting over 99% success.

I've been using rootbsd for about the same time :) The support there is fantastic. The IO problems started when I migrated to a new FreeBSD install with ZFS. UFS2 would have been a better choice for the mail storage, but I wanted it all in one big FS for simplicity.

Each night I'd look at 400 days worth of spam/ham and rebuild each user's DB from scratch. The script would easily process perhaps 40,000 files, taking about 4 hours.

Like I said, on UFS2 it was no problem, but on ZFS it took so much IO that it was affecting other users.

My updated solution is working out quite nicely!
 
Re: [Solved] Dovecot2 Antispam MAILTRAIN backend / SpamAssas

@MrFollies and @wblock@ thanks for the info, I'm going to give 'em a shot.
 
Back
Top