PostgreSQL+Postfix+Nginx+PHP+RoundCube+Dovecot+ SpamAssassin+Clamav+Spamd

osx-addict

Active Member

Thanks: 3
Messages: 174

#27
Ok.. So I got this setup yesterday on my FreeBSD 8-stable system in the following configuration :

host : example.com
jail1 : db.example.com (PostgreSQL 8.4 server)
jail2 : smtp.example.com (Postfix, Dovecot, SpamAssassin, ClamAV -- no Spamd or PF)

I've skipped for now RoundCube and NGIx since I'll leave that for a later chapter once the main mail server is up and running as expected. I had to tweak the SQL a bit to make PostgreSQL happy with it (I dropped a note to KAS about this a bit ago so he can update the thread). I'll admit right now that I had some issues with the Postfix config files -- primarily master.cf. Initially Postfix was complaining about the stuff at the end but after a bit of googling around I found I had a line in the wrong place and moved it -- I believe in the end I disabled the 'scan' line that drives SpamAssassin.. Below are the offending lines from my files for reference :

First file : main.cf (comments removed for brevity sake)
Code:
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
data_directory = /var/db/postfix
mail_owner = postfix
myhostname = smtp.example.com
mydomain = example.com
unknown_local_recipient_reject_code = 550
mynetworks_style = host
debug_peer_level = 2
sendmail_path = /usr/local/sbin/sendmail
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
setgid_group = maildrop
html_directory = /usr/local/share/doc/postfix
manpage_directory = /usr/local/man
sample_directory = /usr/local/etc/postfix
readme_directory = /usr/local/share/doc/postfix
smtpd_recipient_restrictions =
  permit_mynetworks,
  permit_sasl_authenticated,
  reject_non_fqdn_hostname,
  reject_non_fqdn_sender,
  reject_non_fqdn_recipient,
  reject_unauth_destination,
  reject_unauth_pipelining,
  reject_invalid_hostname,
  reject_rbl_client bl.spamcop.net

smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks

virtual_mailbox_base = /mail
virtual_mailbox_maps = pgsql:/usr/local/etc/postfix/pgsql_virtual_mailbox_maps.cf
virtual_mailbox_domains = pgsql:/usr/local/etc/postfix/pgsql_virtual_mailbox_domains.cf
virtual_alias_maps = pgsql:/usr/local/etc/postfix/pgsql_virtual_alias_maps.cf
local_recipient_maps = $virtual_mailbox_maps
virtual_create_maildirsize = yes
virtual_mailbox_extended = yes

virtual_uid_maps = static:26
virtual_gid_maps = static:6

virtual_transport = dovecot

smtpd_delay_reject = yes
smtpd_helo_required = yes


broken_sasl_auth_clients = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = /var/run/dovecot/auth-client
smtpd_sasl_security_options = noanonymous


smtp_use_tls = yes
smtpd_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/ssl/postfix/smtpd.pem
smtpd_tls_cert_file = /etc/ssl/postfix/smtpd.pem
smtpd_tls_CAfile = /etc/ssl/postfix/smtpd.pem
smtpd_tls_loglevel = 0
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

mydestination = localhost.$mydomain, localhost
relay_domains = pgsql:/usr/local/etc/postfix/pgsql_relay_domains.cf
Now for master.cf (again, no comments):
Code:
smtp      inet  n       -       n       -       -       smtpd -o content_filter=spamassassin
#               content_filter = scan:[10.0.1.4]:10025
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
pickup    fifo  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
        -o smtp_fallback_relay=
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache

spamassassin unix  -       n       n       -       -       pipe
   user=nobody argv=/usr/local/bin/spamc -u mailnull -d 10.0.1.4 -e /usr/local/sbin/sendmail -oi -f ${sender} ${recipient}

dovecot    unix -        n       n       -       -       pipe
  flags=DRhu user=mailnull:mail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user}@${nexthop} -n -m ${extension}

scan      unix  -       -       n       -       16      smtp
        -o smtp_send_xforward_command=yes
        -o smtp_enforce_tls=no

10.0.1.4:10026 inet  n -       n       -       16      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks_style=host
        -o smtpd_authorized_xforward_hosts=smtp.example.com

Postfix is fine with this config -- no complaints. However, when started it accepts incoming connections and doesn't appear to do anything with them.. Below is a snippet of the log data showing it running for 3 minutes -- 'top' showed no signs of life from clamav, SpamAssassin or anything else during this period.. Did I miss something -- perhaps related to the lack of running PF + Spamd (the greylisting part)? One thing I need to do is change the IP of the mail jail temporarily to allow me to diagnose these problems -- as soon as I start Postfix, I had about 5 connections within about 3 seconds.. All connections are routed to port 10.0.1.4 from my Cisco router -- if I tell it to use something else then I can diagnose in a less stressed (for me) environment.. :)

Code:
Mar 10 00:46:12 smtp postfix/master[12633]: daemon started -- version 2.7.0, configuration /usr/local/etc/postfix
Mar 10 00:46:13 smtp postfix/smtpd[12636]: connect from unknown[210.245.11.52]
Mar 10 00:46:28 smtp postfix/smtpd[12640]: connect from snt0-omc2-s7.snt0.hotmail.com[65.55.90.82]
Mar 10 00:46:35 smtp postfix/smtpd[12642]: connect from n49d.bullet.mail.sp1.yahoo.com[66.163.169.175]
Mar 10 00:46:43 smtp postfix/smtpd[12643]: connect from n47d.bullet.mail.sp1.yahoo.com[66.163.169.173]
Mar 10 00:46:43 smtp postfix/smtpd[12644]: connect from n47a.bullet.mail.sp1.yahoo.com[66.163.168.141]
Mar 10 00:47:01 smtp postfix/smtpd[12652]: connect from snt0-omc4-s4.snt0.hotmail.com[65.55.90.207]
Mar 10 00:47:53 smtp postfix/smtpd[12657]: connect from e227.en25.com[209.167.231.227]
Mar 10 00:48:10 smtp postfix/smtpd[12659]: connect from dcs-maillist.cs.uiuc.edu[128.174.252.7]
Mar 10 00:48:36 smtp postfix/smtpd[12664]: connect from bay0-omc2-s16.bay0.hotmail.com[65.54.190.91]
Mar 10 00:49:03 smtp postfix/smtpd[12671]: connect from n51c.bullet.mail.sp1.yahoo.com[66.163.168.185]
Mar 10 00:49:03 smtp postfix/smtpd[12670]: connect from bay0-omc1-s20.bay0.hotmail.com[65.54.190.31]
Mar 10 00:49:44 smtp postfix/postfix-script[12692]: stopping the Postfix mail system
Mar 10 00:49:44 smtp postfix/master[12633]: terminating on signal 15
 

halplus

Member

Thanks: 1
Messages: 64

#28
Code:
Apr 10 10:54:47 somehost dovecot: auth(default): prefetch(root@somehost.tld): userdb lookup not possible with only userdb prefetch
An idea?
 

halplus

Member

Thanks: 1
Messages: 64

#31
This is the config

Code:
-- Config start ---------------------
# 1.2.10: /usr/local/etc/dovecot.conf
# OS: FreeBSD 7.0-RELEASE-p11 i386  ufs
protocols: imaps
ssl_cert_file: /etc/ssl/dovecot/cert.pem
ssl_key_file: /etc/ssl/dovecot/key.pem
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
verbose_proctitle: yes
first_valid_uid: 26
last_valid_uid: 26
first_valid_gid: 6
last_valid_gid: 6
mail_privileged_group: mail
mail_uid: mailnull
mail_gid: mail
mail_location: maildir:/mail/%d/%n
mail_plugins: quota imap_quota
imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep
lda:
  postmaster_address: [email]postmaster@example.com[/email]
  sendmail_path: /usr/sbin/sendmail
  mail_plugins: quota
auth default:
  username_format: %Lu
  passdb:
    driver: sql
    args: /usr/local/etc/dovecot-sql.conf
  userdb:
    driver: prefetch
  socket:
    type: listen
    client:
      path: /var/run/dovecot/auth-client
      mode: 432
      user: postfix
      group: mail
    master:
      path: /var/run/dovecot/auth-master
      mode: 384
      user: mailnull
      group: mail
plugin:
  quota: maildir:User quota
  quota_rule: *:storage=1GB
-- Config end ---------------------
I see some postmaster@example.com weirdness but i don't think that's the thing


I remember that when i installed postfix i got this (maybe that's related).

Code:
somehost# chmod mailnull:mail /mail
chmod: invalid file mode: mailnull:mail
I honestly have no idea on what kind of permissions those softwares require and what you were trying to do.

I want to add a constructive coment about the tutorial. It is great and as i can see can be almost blindly followed but you forget at some points to explain what you are doing. I know i should RTFM but ok, my conectivity is incredibly limited (we pay 140 dollars for 80 hours of dialup internet conectivity) So every minute counts. Therefore manuals take unfortunately a second position.

I see that emails return and bounce back to the sender with an error that says something like the user is not existant. Like for instance if they weren't added to the database. HEre is a sample

-- sample start -------------
Code:
This is the mail system at host somehost.tld.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<root@somehost.tld>: user unknown
-- sample end -------------

Also i want to add that i didn't deployed inside a Jail and that i didn't installed a web server, php or roundcube. I am planning to use another solution, partially coded on my own.
What I installed so far is postgres + postfix + dovecot on top of Freebsd. (I landed in this great TUT becuse those where my choices). And i plan to add dkimproxy to sign messages. As a matters of fact i istalled it already but haven't configured postfix to use it, in order to not add extra configuration noise. I plan some time in the future to add spamassasin as you did, as well as clamav and maybe greylisting and SPF checking. But that is future polishment.
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#32
halplus said:
I remember that when i installed postfix i got this (maybe that's related).

Code:
somehost# chmod mailnull:mail /mail
chmod: invalid file mode: mailnull:mail
That was bug in my thread, I fixed it.
It was supposed to be chown

halplus said:
I honestly have no idea on what kind of permissions those softwares require and what you were trying to do.
You obviously lack basic Unix knowledge

halplus said:
I want to add a constructive coment about the tutorial. It is great and as i can see can be almost blindly followed but you forget at some points to explain what you are doing. I know i should RTFM but ok, my conectivity is incredibly limited (we pay 140 dollars for 80 hours of dialup internet conectivity) So every minute counts. Therefore manuals take unfortunately a second position.
If I had like 48h / day, I'd be glad to provide extra details, but I have lot's and lot's of stuff to do for University, and currently my time is very limited.
Also, before you run your mail server you should understand how it works. Otherwise when problems occur, you won't be able to fix them on your own

halplus said:
I see that emails return and bounce back to the sender with an error that says something like the user is not existant. Like for instance if they weren't added to the database. HEre is a sample

-- sample start -------------
Code:
This is the mail system at host somehost.tld.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<root@somehost.tld>: user unknown
-- sample end -------------
Maybe this is because you didn't # chown mailnull:mail /mnt see Update section
 

halplus

Member

Thanks: 1
Messages: 64

#34
You obviously lack basic Unix knowledge
I don't think that dovecot configuration can be considered "Basic Unix Knowledge".

When I said "those softwares" I meant dovecot and postfix not chmod which is only one ;). (Unless i am a supergenious and tried to let you realize by yourself that i knew that we were talking about chown too ;) in wich case we could say "those" so with that word i manipulated your head into thinking that, so you then could say that you made a mistake on your own without me having you to tell you). I just tried that line, saw that was wrong and didn't recognized that it was supposed to be chown. Simple. That kind of stuff happens when you work under heavy pressure during 5 hours the entire night because over the day the poor line couples with your neighbor and when she speaks (lots of sh..) takes down your connection which of course is not nice when you work with root in a server on the other side of the sea and you really don't want to break it. In any case we won a better tutorial. Now is fixed! Right?


If I had like 48h / day, I'd be glad to provide extra details, but I have lot's and lot's of stuff to do for University, and currently my time is very limited.
Well was just an idea, since i saw that you solicited opinions. And don't worry I studied computer science too, so I know what you mean. I remember the discrete Math teacher was a Nazi that putted Informatics Olympiad questions in the final tests. "Ahh.. they are so easy" pretending he was the smartest ass. Used to say "see... trivial". (yes well now that you see the proof couldn't be easier) Jaj, some students were smarter and nobody left the room before 8 hours in the test. I think i would love to see him doing one of those tests sweating a lot and telling everyone "see... trivial" with a blank page while we think "sure let's see if you can do what we could". Ajajajajja.


Also, before you run your mail server you should understand how it works. Otherwise when problems occur, you won't be able to fix them on your own.
Nahh I plan to drop all of the software over time and write highly scalable java based alternatives
with reduced complexity and also store mail directly inside the database (whether conventional database on top of distributed cluster in async multimaster mode or hbase on top of hadoop), and not in those mbox or the other non_portable_outside_unix_non_scalable standard. Anyway right now i need this thingie. I am not that big and is likely to take a while before I conquer the entire planet/universe and beat yahoo mail and gmail :p. By then they will be so poor and i will have so many billions stolen from them that I will not care to hand you a few to fix such a little thing for me ;). Just joking around. I believe that i will be able. Right now i need to cancel the debt, i still do not take down yahoo ;).


Maybe this is because you didn't # chown mailnull:mail /mnt see Update section
I believe you mean /mail. No, that made no difference at all. Is something else. I think dovecot is not properly configured or something.


Another thing. It seems to me that you changed email storage from /var/mail to /mail right? Any specific purpose on that? I want to take things back there instead of showing in root dir, i simply don't want to add extra noise ATM.


show /usr/local/etc/dovecot-sql.conf

Code:
somehost# cat /usr/local/etc/dovecot-sql.conf

... (comments)

driver = pgsql
connect = host=/tmp dbname=mail user=dovecot password=SOMEPASSWORD
default_pass_scheme = PLAIN-MD5

password_query = \
  SELECT username, domain, password, '*:bytes=' || quota || 'M' AS userdb_quota_
rule \
  FROM mailbox WHERE username = '%n' AND domain = '%d' AND active = true
somehost#
I don't see user_query or tables with userdb_ prefixes or uids and gids like in

http://wiki.dovecot.org/UserDatabase/Prefetch

Here http://wiki.dovecot.org/AuthDatabase/SQL you can read:

If you're using a single UID and GID for all users, you can use a SQL query such as:

user_query = SELECT 500 AS uid, 500 AS gid, home FROM users WHERE userid = '%n'
seems to me that's the way should be done since i do not have such users in my system therefore no uid or gid.

so i added in dovecot-sql.conf (and the consequent line in dovecot.conf) this line:

Code:
user_query = SELECT username,26 AS uid,6 AS gid FROM mailbox WHERE username = '%n'
However now i am getting this:
Code:
Apr 11 12:15:48 somehost dovecot: deliver(root@somehost.tld): msgid<400669.91433.qm@web65702.mail.ac4.yahoo.com>: saved mail to INBOX
...
Apr 11 12:26:34 somehost dovecot: imap-login: Aborted login (auth failed, 4 attempts): user=<root>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS
Note that i haven't added any userdb_ prefix. However i don't know exactly how i should do that if ever.
 

BeastieBoy

Member

Thanks: 1
Messages: 80

#35
Excellent tutorial!

And you did well not to get into the details as they belong into other threads.

Concerning the Postgresql post initialization, I got the following message:
Success. You can now start the database server using:

Code:
    /usr/local/bin/postgres -D /usr/local/pgsql/data
or
Code:
    /usr/local/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start[/QUOTE]
Your /usr/local/etc/rc.d/postgresql start does work, I was just wondering why your command is different from what's output?
 

BeastieBoy

Member

Thanks: 1
Messages: 80

#36
I think you're missing some Postgresql details. This database is a beast, and knowing how to configure it, create users, databases is important. Just to "to get started" command should be enough.
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#38
Thanks for feedback...
I'll try to fix this tomorrow..... if I can [lately I have problems posting to forum]
If I can't I will ask mods to fix this for me
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#39
BeastieBoy said:
Concerning the Postgresql post initialization, I got the following message:

Your /usr/local/etc/rc.d/postgresql start does work, I was just wondering why your command is different from what's output?
Not sure right now... cause lot of time have passed, and I did many things diferently each time I rebuild system :)

Also I wrote this tutorial, after I actually implemented everything, and I tried to keep it generic, so no wonder some things may be incorrect... but at least I think it's a good starting point... [at least every time I need, I look for help here :D ]
 

BeastieBoy

Member

Thanks: 1
Messages: 80

#43
In the configuration of dovecot, in the socket listen { ... }, you specify a client path in /var/run/dovecot/auth-client, but I think it's wrong, it should be in /var/spool/postfix/private, because user postfix cannot create a file under /var/run/dovecot unless I missed a step.
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#44
BeastieBoy said:
In the configuration of dovecot, in the socket listen { ... }, you specify a client path in /var/run/dovecot/auth-client, but I think it's wrong, it should be in /var/spool/postfix/private, because user postfix cannot create a file under /var/run/dovecot unless I missed a step.
in postifx /usr/local/etc/main.cf I have
Code:
smtpd_sasl_path = /var/run/dovecot/auth-client
and it works.... {I'm not chrooting.... Probably will fix this one day}
 

BeastieBoy

Member

Thanks: 1
Messages: 80

#45
Hi,

I decided to add support for smtps. And guess what, ItFailsâ„¢.

imaps connection works so I think I'm not too far from getting it to work.

Here is what I have got in /var/log/maillog:
Code:
postfix/smtpd[16773]: SSL_accept error from ...
Any idea what could cause that? The error isn't very verbose.
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#46
check in main.cf
Code:
smtp_use_tls = yes
smtpd_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/ssl/postfix/smtpd.pem
smtpd_tls_cert_file = /etc/ssl/postfix/smtpd.pem
smtpd_tls_CAfile = /etc/ssl/postfix/smtpd.pem
smtpd_tls_loglevel = 0
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
in master.cf
Code:
smtps     inet  n       -       n       -       -       smtpd
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o message_size_limit=26214400
maybe you're missing something important :D

also check file permissions on your certificate
 

BeastieBoy

Member

Thanks: 1
Messages: 80

#47
Speaking about certificate. For imaps access I get an annoying warning message about a self-signed certificate. How to get rid of that? Do you have that problem too?
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#48
well, you can either pay some trusted authority to sign it for you

https://www.startssl.com/ offers free signing of certificates (but I haven't read license terms)

I import my certificate to apps (opera for example [using opera M2 mail client])
after that there is no annoying message about self-signed certificates :D
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 686
Messages: 4,611

#50
kpa said:
Or create your own CA and sign your application certificates yourself :p
Which will
BeastieBoy said:
For imaps access I get an annoying warning message about a self-signed certificate. How to get rid of that? Do you have that problem too?
 
Top