Solved Qmail will not receive mail from outside

Rob215x · Sep 24, 2019

I have a Qmail installation mostly working, but I'm not receiving any mail. I'm looking for suggestions on how I can trace where the problem is.

I'm trying to get Qmail configured on a test server at my office, before I mess with my production server. My FreeBSD is:

Code:

FreeBSD 11.2-RELEASE-p9 (GENERIC) #0: Tue Feb  5 15:30:36 UTC 2019

My ports are all up to date. I'm running Apache 2.4.41, and PHP 7.1.32 (not that Apache and PHP affect Qmail directly, but I can open php pages for the 2 subdomains running on the test server)

My test server is connected to an ASUS router in my office, which is connected to a local internet provider. My server has an IP address of 192.168.0.154.
The router is configured to forward all of the relevant ports to the test server.

- I have 2 subdomains configured, with the first being the server's hostname. I don't need to give those out here right?
- I have SSL certs from Letsencrypt configured and working for both subdomains. (I can pull up default html pages in a browser and see the green lock).
- I made a simple web form with PHP and messages from that form ARE being delivered to my external Gmail address.

Here are the qmail and related services running:

Code:

# svstat /service/* /service/*/log
/service/dovecot: up (pid 65049) 1271 seconds
/service/qmail-send: up (pid 61408) 1271 seconds
/service/qmail-smtpd: up (pid 62824) 1271 seconds
/service/qmail-smtpd-ssl: up (pid 63710) 1271 seconds
/service/qmail-smtpd-tls: up (pid 65207) 1271 seconds
/service/qmail-updater: up (pid 59963) 1271 seconds
/service/dovecot/log: up (pid 66430) 1271 seconds
/service/qmail-send/log: up (pid 62347) 1271 seconds
/service/qmail-smtpd-ssl/log: up (pid 64329) 1271 seconds
/service/qmail-smtpd-tls/log: up (pid 65440) 1271 seconds
/service/qmail-smtpd/log: up (pid 65321) 1271 seconds
/service/qmail-updater/log: up (pid 60650) 1271 seconds

I can do this:

Code:

# echo To: fake-address@gmail.com | /var/qmail/bin/qmail-inject

and a blank message gets delivered to my gmail account

- I have added the two subdomains and postmaster accounts and they appear in the list when I do the "mkvalidrcptto" command.
- I can connect to the postmaster account using Thunderbird at home and then running the following command shows the connection I made from Thunderbird:

Code:

# netstat -anp tcp
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address          Foreign Address        (state)
tcp4       0      0 192.168.0.154.143      xx.xx.xx.xx.43236    ESTABLISHED
tcp4       0      0 192.168.0.154.143      xx.xx.xx.xx.43234    ESTABLISHED
tcp4       0     44 192.168.0.154.22       xx.xx.xx.xx.49704    ESTABLISHED
tcp6       0      0 *.21                   *.*                    LISTEN
tcp4       0      0 *.21                   *.*                    LISTEN
tcp4       0      0 *.443                  *.*                    LISTEN
tcp6       0      0 *.443                  *.*                    LISTEN
tcp4       0      0 *.80                   *.*                    LISTEN
tcp6       0      0 *.80                   *.*                    LISTEN
tcp4       0      0 *.22                   *.*                    LISTEN
tcp6       0      0 *.22                   *.*                    LISTEN
tcp4       0      0 127.0.0.1.3306         *.*                    LISTEN
tcp4       0      0 127.0.0.1.9000         *.*                    LISTEN
tcp4       0      0 *.993                  *.*                    LISTEN
tcp4       0      0 *.143                  *.*                    LISTEN
tcp4       0      0 *.995                  *.*                    LISTEN
tcp4       0      0 *.110                  *.*                    LISTEN
tcp4       0      0 192.168.0.154.25       *.*                    LISTEN
tcp4       0      0 192.168.0.154.587      *.*                    LISTEN
tcp4       0      0 192.168.0.154.465      *.*                    LISTEN

I have replaced my home IP address with xx.xx.xx.xx but you can see both the port 143 connections and my port 22 SSH connection

I also get the following:

Code:

# telnet 192.168.0.154 25
Trying 192.168.0.154...
Connected to pulp7.
Escape character is '^]'.
421 unable to read controls (#4.3.0)
Connection closed by foreign host.

# telnet 192.168.0.154 143
Trying 192.168.0.154...
Connected to pulp7.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] MAILHOST IMAP READY.
...
* BYE Logging out
a OK Logout completed (0.001 + 0.000 secs).
Connection closed by foreign host.

So I CAN connect to port 143 locally but not 25. Why?

I've checked the qmail logs. I don't see any error messages. I've checked the permissions of the files and folders and they all look right. Obviously I'm missing something. Everything seems to be MOSTLY working, but I'm not getting any incoming messages.

The emails I send TO my test server from my Gmail account get returned with the same "421 unable to read controls (#4.3.0)" message as when I try to telnet to port 25 locally.

I'm stuck. What should I check next or what tests can I run next?

thanks!

VladiBG · Sep 24, 2019

is your /var/qmail/control readable by qmail user?
paste the run script of qmail-smtpd

Rob215x · Sep 24, 2019

VladiBG said:
is your /var/qmail/control readable by qmail user?
paste the run script of qmail-smtpd

Here's the control folder:

Code:

drwxr-xr-x   2 root    qmail      512 Sep 23 23:53 control

Here's qmail-smtpd/run:

Code:

#!/bin/sh
#
# service-qmail-smtpd-run (formerly "run.smtp" and "run.smtp.sslserver")
# John Simpson <jms1@jms1.net> 2003-07-05 to 2008-03-24
#
# Generic daemontools "run" script for qmail "smtp" or "smtpssl" service.
#
# Documentation: http://qmail.jms1.net/scripts/service-qmail-smtpd-run.shtml
#
###############################################################################
#
# Copyright (C) 2003-2007 John Simpson.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License, version 2, as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
# or visit http://www.gnu.org/licenses/gpl.txt
#
###############################################################################

exec 2>&1
VQ="/var/qmail"
PATH="$VQ/bin:/usr/local/bin:/usr/bin:/bin"
QUSER=qmaild
LOCAL=`head -1 $VQ/control/me`

###############################################################################
#
# options for tcpserver/sslserver

IP=192.168.0.154
PORT=25
SSL=0
SSL_CERT="$VQ/control/servercert.pem"
SMTP_CDB="/etc/tcp/smtp.cdb"
MAX=30

# these require the "tcpserver limits" patch for ucspi-tcp, available here:
# http://linux.voyager.hr/ucspi-tcp/

#MAXLOAD=750
#MAXCONNIP=2
#MAXCONNC=5
#DIEMSG="421 $LOCAL Service temporarily unavailable"

# my newer version of the tcpserver limits patch allows you to specify
# individual DIEMSG values for each policy.
# http://qmail.jms1.net/ucspi-tcp/

#DIEMSG_MAXLOAD="421 $LOCAL Server busy, try again later."
#DIEMSG_MAXCONNIP="421 $LOCAL Too many connections from your IP."
#DIEMSG_MAXCONNC="421 $LOCAL Too many connections from your network."

###############################################################################
#
# options for programs which run before qmail-smtpd

#RBLSMTPD_PROG="/usr/local/bin/rblsmtpd"
#RBL_GOOD=""
#RBL_BAD=""
#RBL_BAD="zen.spamhaus.org dnsbl.njabl.org dnsbl.sorbs.net bl.spamcop.net"

#GREYLIST="jgreylist"

#JGREYLIST_DIR="$VQ/jgreylist"
#JGREYLIST_NOREV=1
#JGREYLIST_BY_IP=0
#JGREYLIST_HOLDTIME=120
#JGREYLIST_LOG=1
#JGREYLIST_LOG_PID=1
#JGREYLIST_LOG_SMTP=0
#JGREYLIST_TIMEOUT=60
#JGREYLIST_LIMIT=0

#RECORDIO="recordio"

###############################################################################
#
# options for qmail-smtpd itself

SMTPD="/var/qmail/bin/qmail-smtpd"
SMTPGREETING="$LOCAL NO UCE"
GREETDELAY=0
#DROP_PRE_GREET=1
FORCE_TLS=0
DENY_TLS=0
MFCHECK=0
#MAXRCPT=100
#RELAYREJ=1
QMAILSMTPD_LOG_MAIL=1
QMAILSMTPD_LOG_RCPT=1
#QMAILSMTPD_HELP_VERSION=1

###############################################################################
#
# options pertaining to the AUTH command.

AUTH=0
REQUIRE_AUTH=0
ALLOW_INSECURE_AUTH=0

# if using the AUTH_CDB method
#AUTH_CDB="$VQ/control/auth.cdb"

# if using the CHECKPW method
CHECKPW="/usr/home/vpopmail/bin/vchkpw"
TRUE=`which true`

# to change the environment whenever somebody authenticates
#AUTH_SET_MFCHECK=0
#AUTH_SET_MAXRCPT=0
#AUTH_SET_DATABYTES=0
#AUTH_SET_SPFBEHAVIOR=1
#AUTH_SET_VALIDRCPTTO_LIMIT=10
#AUTH_SET_VALIDRCPTTO_LOG=1
#AUTH_SET_SPF_LOG=1
#AUTH_SET_RELAYREJ=0
#AUTH_SET_VALIDRCPTTO_CDB=""
#AUTH_SET_QMAILSMTPD_LOG_MAIL=1
#AUTH_SET_QMAILSMTPD_LOG_RCPT=1
#AUTH_SET_QMAILSMTPD_HELP_VERSION=1

###############################################################################
#
# options pertaining to the "validrcptto.cdb" mechanism.
# see http://qmail.jms1.net/patches/validrcptto.cdb.shtml for details.

VALIDRCPTTO_CDB="$VQ/control/validrcptto.cdb"
#VALIDRCPTTO_LIMIT=10
VALIDRCPTTO_LOG=2

###############################################################################
#
# options pertaining to the SPF mechanism.

SPFBEHAVIOR=0
SPF_LOG=1
SPF_BLOCK_PLUS_ALL=1

###############################################################################
#
# options pertaining to the Domainkeys mechanism.
# this requires an add-on patch.

#DOMAINKEYS=0
#DKVERIFY=DEfGhIJK
#AUTH_SET_DKSIGN=/etc/domainkeys/%/default

###############################################################################
#
# options for programs which run after qmail-smtpd

# if you are using simscan...

#QMAILQUEUE="$VQ/bin/simscan"
NOP0FCHECK=1
#SIMSCAN_DEBUG=0
#SIMSCAN_DEBUG_FILES=0

# if you are using qmail-scanner, un-comment ONE of these lines.

#QMAILQUEUE="$VQ/bin/qmail-scanner-queue"

# if you're using some other qmail-queue replacement, add your own line here
# with the correct value.


###############################################################################
###############################################################################
###############################################################################
#
# THERE SHOULD BE NO NEED TO CHANGE ANYTHING BELOW THIS LINE. of course, the
# script is on your system and you're free to edit it however you want, but
# changing things below this point may cause strange things to happen. make
# sure you understand what you're doing if you change anything below...

QDUID=`id -u $QUSER`
QDGID=`id -g $QUSER`

if [ -z "$IP" -o "$IP" = "unset" ]
then
    echo "The IP variable is not set in the run script. Cannot start."
    sleep 5
    exit 1
fi

if [ -z "$QDUID" -o -z "$QDGID" -o -z "$MAX" -o -z "$LOCAL" \
    -o -z "$SSL" -o -z "$AUTH" ]
then
    echo "One of the variables QDUID, QDGID, MAX, LOCAL, SSL, or AUTH"
    echo "is not set in the run script. Cannot start."
    sleep 5
    exit 1
fi

if [ ! -f $VQ/control/rcpthosts ]
then
    echo Creating emtpy $VQ/control/rcpthosts file to prevent open relay.
    touch $VQ/control/rcpthosts
    chmod 644 $VQ/control/rcpthosts
fi

if [ "$SSL" = "1" ]
then
    if ! which sslserver > /dev/null 2>&1
    then
        echo ERROR: sslserver not found in PATH [$PATH]
        exit 1
    fi

    if [ ! -f $SSL_CERT ]
    then
        echo ERROR: $SSL_CERT does not exist
        exit 1
    fi

    export CERTFILE=${SSL_CERT}
    export KEYFILE=""
    export DHFILE=""

    SCMD="sslserver -e"
else
    if [ -n "$SSL_CERT" ]
    then
        export TLS_SERVER_CERT=${SSL_CERT}
    fi

    SCMD="tcpserver"
fi

if [ "$IP" = "127.0.0.1" ]
then
    export RELAYCLIENT=""
    RBLSMTPD_PROG=""
    ACMD=""
elif [ -z "${SMTP_CDB:-}" ]
then
    ACMD=""
else
    if [ ! -f "$SMTP_CDB" ]
    then
        echo ERROR: $SMTP_CDB does not exist
        exit 1
    fi

    ACMD="-x $SMTP_CDB"
fi

if [ "$AUTH" = "1" ]
then
    if [ -n "$AUTH_CDB" ]
    then
        if [ ! -f $AUTH_CDB ]
        then
            echo ERROR: AUTH_CDB file [$AUTH_CDB] does not exist
            exit 1
        fi

        export AUTH_CDB
        ARGS=""
    elif [ -n "$CHECKPW" ]
    then
        if [ ! -f $CHECKPW ]
        then
            echo ERROR: $CHECKPW [$CHECKPW] program does not exist
            exit 1
        fi

        if [ -z "$LOCAL" ]
        then
            echo ERROR: LOCAL is not set in the run script
            exit 1
        elif [ -z "$TRUE" ]
        then
            echo ERROR: TRUE is not set in the run script
            exit 1
        elif [ ! -e $TRUE ]
        then
            echo ERROR: $TRUE [$TRUE] is not an executable
            exit 1
        fi

        ARGS=" $LOCAL $CHECKPW $TRUE"
    else
        echo ERROR: AUTH=1 but no AUTH_CDB or CHECKPW
        exit 1
    fi
else
    ARGS=""
    AUTH_CDB=""
fi

########################################
# make RBL command (if needed)

#RBLCMD2="/usr/local/bin/spamdyke -f /usr/local/etc/spamdyke.conf"

RBLCMD=""

if [ -n "$RBLSMTPD_PROG" ]
then
    if [ -n "$RBL_GOOD" ]
    then
        for name in $RBL_GOOD
        do
            RBLCMD="$RBLCMD -a $name"
        done
    fi

    if [ -n "$RBL_BAD" ]
    then
        for name in $RBL_BAD
        do
            RBLCMD="$RBLCMD -r $name"
        done
    fi

    if [ -n "$RBLCMD" ]
    then
        RBLCMD="$RBLSMTPD_PROG -t0 $RBLCMD"
    fi
fi

########################################
# make domainkeys command (if needed)

if [ "$DOMAINKEYS" = "1" ]
then
    if [ -f "$VQ/bin/qmail-dk" ]
    then
        if [ -n "$QMAILQUEUE" ]
        then
            export DKQUEUE="$QMAILQUEUE"
        fi
        export AUTH_UNSET_DKVERIFY=1
        export QMAILQUEUE="$VQ/bin/qmail-dk"

        if [ -n "$DKVERIFY" ] ; then export DKVERIFY ; fi
        if [ -n "$DKSIGN"   ] ; then export DKSIGN   ; fi
    else
        echo ERROR: $VQ/bin/qmail-dk not found, cannot use domainkeys
    fi
fi

########################################
# handle variables which may not have been set, but need to exist even
# if they contain blank values

if [ -z "$RECORDIO" ]
then
    RECORDIO=""
fi

if [ -z "$GREYLIST" ]
then
    GREYLIST=""
fi

########################################
# do the deed

for n in SSL                    \
    MAXLOAD                    \
    MAXCONNIP                \
    MAXCONNC                \
    DIEMSG                    \
    DIEMSG_MAXLOAD                \
    DIEMSG_MAXCONNIP            \
    DIEMSG_MAXCONNC                \
    JGREYLIST_DIR                \
    JGREYLIST_NOREV                \
    JGREYLIST_BY_IP                \
    JGREYLIST_HOLDTIME            \
    JGREYLIST_LOG                \
    JGREYLIST_LOG_PID            \
    JGREYLIST_LOG_SMTP            \
    JGREYLIST_TIMEOUT            \
    JGREYLIST_LIMIT                \
    SMTPGREETING                \
    GREETDELAY                \
    DROP_PRE_GREET                \
    FORCE_TLS                \
    DENY_TLS                \
    MFCHECK                    \
    MAXRCPT                    \
    RELAYREJ                \
    QMAILSMTPD_LOG_MAIL            \
    QMAILSMTPD_LOG_RCPT            \
    QMAILSMTPD_HELP_VERSION            \
    REQUIRE_AUTH                \
    ALLOW_INSECURE_AUTH            \
    AUTH_CDB                \
    AUTH_SET_MFCHECK            \
    AUTH_SET_MAXRCPT            \
    AUTH_SET_DATABYTES            \
    AUTH_SET_SPFBEHAVIOR            \
    AUTH_SET_VALIDRCPTTO_LIMIT        \
    AUTH_SET_VALIDRCPTTO_LOG        \
    AUTH_SET_SPF_LOG            \
    AUTH_SET_RELAYREJ            \
    AUTH_SET_VALIDRCPTTO_CDB        \
    AUTH_SET_QMAILSMTPD_LOG_MAIL        \
    AUTH_SET_QMAILSMTPD_LOG_RCPT        \
    AUTH_SET_QMAILSMTPD_HELP_VERSION    \
    VALIDRCPTTO_CDB                \
    VALIDRCPTTO_LIMIT            \
    VALIDRCPTTO_LOG                \
    SPFBEHAVIOR                \
    SPF_LOG                    \
    SPF_BLOCK_PLUS_ALL            \
    DKVERIFY                \
    AUTH_SET_DKSIGN                \
    QMAILQUEUE                \
    NOP0FCHECK                \
    SIMSCAN_DEBUG                \
    SIMSCAN_DEBUG_FILES
do
    # note: not 100% sure "eval" works under old-school /bin/sh
    eval "if [ -n \"\$$n\" ];then echo \"$n=\\\"\$$n\\\"\";export $n;fi"
done

CMD="$SCMD -vR -l $LOCAL -c $MAX -u $QDUID -g $QDGID $ACMD $IP $PORT"
CMD="$CMD $RBLCMD $RBLCMD2 $GREYLIST $RECORDIO $SMTPD $ARGS"

echo "command-line: exec $CMD 2>&1"
exec $CMD 2>&1

########################################
# this will only be reached if the exec fails

echo ERROR: command did not run correctly
exit 1
#CMD="$CMD $RBLCMD $GREYLIST $RECORDIO /usr/local/bin/spamdyke -f /usr/local/etc

VladiBG · Sep 24, 2019

check the permissions of validrcptto.cdb file
/var/qmail/control/validrcptto.cdb

edit:
also check
/etc/tcp/smtp.cdb
it should be readable by anyone 644

Rob215x · Sep 24, 2019

VladiBG said:
check the permissions of validrcptto.cdb file
/var/qmail/control/validrcptto.cdb

edit:
also check
/etc/tcp/smtp.cdb
it should be readable by anyone 644

/ect/tcp/smtp.cdb is good. permissions are correct.

But I think you may have discovered the problem. If I do:

Code:

# mkvalidrcptto

it works and shows the list of valid IP addresses. If I add a user with vpopmail, it updates the list. Since it works fine, I never bothered to look for the file. BUT there is no /var/qmail/control/validrcptto.cdb! Where is it?

So the "onchange" patch is working, and mkvalidrcptto works, but its not generating the validrcptto.cdb file.
thoughts?

Rob215x · Sep 25, 2019

So, I generated the validrcptto.cdb file with

Code:

mkvalidrcptto -c validrcptto.cdb

And now, in a terminal I can do:

Code:

# telnet 192.168.0.154 25
Trying 192.168.0.154...
Connected to pulp7.
Escape character is '^]'.
220 pulp7.mydomain.com NO UCE ESMTP

So I tried sending a new email to my server from my Gmail account. But it didn't go through either. This time, instead of the previous "421 unable to read controls (#4.3.0)" error, Gmail reported the following: "read error: generic::failed_precondition: read error (0): error"

I feel like I'm a step closer at least. I'm going to keep playing with it.

If anyone has more suggestions for how I can trace this problem to the source, that would be awesome!

thanks

VladiBG · Sep 25, 2019

Try to send the email using telnet and see what is the exact error. Also check the log file in smtpd folder it should be located under /var/log/qmail/qmail-smtpd

Rob215x · Sep 25, 2019

VladiBG said:
Try to send the email using telnet and see what is the exact error. Also check the log file in smtpd folder it should be located under /var/log/qmail/qmail-smtpd

I'm having trouble connecting to my server (at my office) from home using port 25. It times out. I'm not sure how to check if port 25 is being blocked by my home ISP, my office ISP, my office router, etc. But it looks like Google connects for a second and then is disconnected with a status 11...

recent entries in qmail-smtpd log:

Code:

2019-09-25 14:48:54.197160500 tcpserver: status: 1/30
2019-09-25 14:48:54.197447500 tcpserver: pid 65144 from 209.85.219.194
2019-09-25 14:48:54.198810500 tcpserver: ok 65144 pulp7.mydomain.com:192.168.0.154:25 mail-yb1-f194.google.com:209.85.219.194::46362
2019-09-25 14:48:54.288540500 tcpserver: end 65144 status 11
2019-09-25 14:48:54.288543500 tcpserver: status: 0/30
2019-09-25 15:40:46.327330500 tcpserver: status: 1/30
2019-09-25 15:40:46.327704500 tcpserver: pid 1706 from 209.85.161.68
2019-09-25 15:40:46.344460500 tcpserver: ok 1706 pulp7.mydomain.com:192.168.0.154:25 mail-yw1-f68.google.com:209.85.161.68::38035
2019-09-25 15:40:46.437756500 tcpserver: end 1706 status 11
2019-09-25 15:40:46.437758500 tcpserver: status: 0/30
2019-09-25 17:54:42.222811500 tcpserver: status: 1/30
2019-09-25 17:54:42.223088500 tcpserver: pid 33252 from 172.105.4.227
2019-09-25 17:54:42.225867500 tcpserver: ok 33252 pulp7.mydomain.com:192.168.0.154:25 protoscan.ampereinnotech.com:172.105.4.227::10644
2019-09-25 17:54:55.723530500 tcpserver: end 33252 status 256
2019-09-25 17:54:55.723531500 tcpserver: status: 0/30
2019-09-25 17:54:55.771356500 tcpserver: status: 1/30
2019-09-25 17:54:55.771600500 tcpserver: pid 38172 from 172.105.4.227
2019-09-25 17:54:55.772819500 tcpserver: ok 38172 pulp7.mydomain.com:192.168.0.154:25 protoscan.ampereinnotech.com:172.105.4.227::35478
2019-09-25 17:54:55.870958500 tcpserver: end 38172 status 256
2019-09-25 17:54:55.870960500 tcpserver: status: 0/30

I can connect to my server using telnet with port 587 but I don't know how to use STARTTLS from telnet to send a message.

I'm currently trying to Google how to do that.

thanks for your help so far!

Rob215x · Sep 26, 2019

I just drove to my office, set up a custom port that forwards to 25 on the test server so I can try telnet again from home.

While I was here at the office. I tried using my local network to "telnet 192.168.0.154 25" and sent an email to my server from my laptop (192.168.0.198). It worked and the qmail-smptd log shows:

Code:

2019-09-25 19:00:44.835716500 tcpserver: status: 1/30
2019-09-25 19:00:44.836095500 tcpserver: pid 90549 from 192.168.0.198
2019-09-25 19:00:44.837327500 tcpserver: ok 90549 pulp7.mydomain.com:192.168.0.154:25 network2012:192.168.0.198::51997
2019-09-25 19:01:11.579680500 qmail-smtpd[90549]: MAIL FROM:<myaddress@somewhere.com>
2019-09-25 19:01:41.325103500 qmail-smtpd[90549]: RCPT TO:<postmaster@pulp7.mydomain.com>
2019-09-25 19:01:41.325105500 qmail-smtpd[90549]: validrcptto [192.168.0.198] trying: postmaster@pulp7.mydomain.com
2019-09-25 19:01:41.327129500 qmail-smtpd[90549]: validrcptto [192.168.0.198] found: postmaster@pulp7.mydomain.com
2019-09-25 19:03:55.633581500 tcpserver: end 90549 status 0
2019-09-25 19:03:55.633583500 tcpserver: status: 0/30

I will test again from home (using the custom port) when I return home later tonight.

By the way, could it be a problem if I did not compile ucspi-tcp with the RSS_DIFF patch? Because I did not.

thanks!

Rob215x · Sep 26, 2019

I just returned home. I tried "telnet xx.xx.xx.xx nnnnn" where nnnnn is the custom port I set the router at my office to forward to port 25.

It worked just like it did at my office!! Here's the qmail-smtpd log:

Code:

2019-09-25 23:44:38.659169500 tcpserver: status: 1/30
2019-09-25 23:44:38.659630500 tcpserver: pid 96239 from xx.xx.xx.xx
2019-09-25 23:44:38.661172500 tcpserver: ok 96239 pulp7.mydomain.com:192.168.0.154:25 c-xx-xx-xx-xx.hsd1.fl.comcast.net:xx.xx.xx.xx::49190
2019-09-25 23:45:22.171768500 qmail-smtpd[96239]: MAIL FROM:<myaddress@somewhere.com>
2019-09-25 23:45:46.359116500 qmail-smtpd[96239]: RCPT TO:<postmaster@pulp7.mydomain.com>
2019-09-25 23:45:46.359119500 qmail-smtpd[96239]: validrcptto [xx.xx.xx.xx] trying: postmaster@pulp7.mydomain.com
2019-09-25 23:45:46.359120500 qmail-smtpd[96239]: validrcptto [xx.xx.xx.xx] found: postmaster@pulp7.mydomain.com
2019-09-25 23:46:48.699349500 tcpserver: end 96239 status 0
2019-09-25 23:46:48.699350500 tcpserver: status: 0/30

But this raises more questions...

Is the ISP at my office blocking port 25?
If they are blocking port 25, then how come I see "mail-yb1-f194.google.com" trying to connect on port 25 in my qmail-smtpd logs?
Why is google getting status 11 but my telnet message from home worked?
How can I trace this further?

thanks!!

VladiBG · Sep 26, 2019

You can use https://mxtoolbox.com/diagnostic.aspx

Rob215x · Sep 26, 2019

Just tried it and it looks good...

I don't think my ISP is going to give me a PTR record. But why would I need one to receive emails from my Gmail address?

Rob215x · Sep 26, 2019

Here's an update, and maybe it will help shed some light on what's going on?

Just to summarize, my test server is located at my office, in the next town over. I use one ISP for internet there and I have Comcast (a differerent ISP) at home.

At home, I was just able to connect to my postmaster email account with Thunderbird on my Macbook and with Outlook 2003 (running in a Windows 7 VM) on my PC (Manjaro). I was able to see all of the messages in my INBOX on both machines.

I was also able to SEND an email through my test server, to my Gmail address!

Outlook 2003 did not have an option for TLS so I used port 465 and SSL. It worked fine and the message appeared in my Gmail seconds later.

In Thunderbird, I could not get port 587 with TLS to work. It works fine with 465 and SSL though.

Finally, I used Thunderbird to send an email from postmaster TO the same postmaster account, and yes, it showed up immediately in my test server INBOX.

I'm still not receiving emails from Google. They get returned with "read error: generic::failed_precondition: read error (0): error"
Could I have a problem with TLS if Google is trying to use TLS on port 25?

Thanks

SirDice · Sep 26, 2019

Rob215x said:
At home, I was just able to connect to my postmaster email account with Thunderbird on my Macbook and with Outlook 2003 (running in a Windows 7 VM) on my PC (Manjaro). I was able to see all of the messages in my INBOX on both machines.

This uses POP3 or IMAP and is handled by dovecot.

Rob215x · Sep 26, 2019

SirDice said:
This uses POP3 or IMAP and is handled by dovecot.

Yes, that's true, and I'm using IMAP port 143 to retrieve the messages. But my server still does not receive any emails from the outside world. Yet, I can send a mail from home, through my server, to my Gmail account. I'm stuck.

VladiBG · Sep 26, 2019

Try to send the email not from gmail. They server may be in blacklist.

Rob215x · Sep 27, 2019

VladiBG said:
Try to send the email not from gmail. They server may be in blacklist.

Interesting. I checked a couple of the IP addresses that Google is using to deliver mail and they ARE listed in several RBLs:

209.85.219.194 Listed in 3 RBLs!!!
209.85.161.68 Listed in 3 RBLs!!!

So next, I tried sending an email from one of my client's servers (I built their web site and set up an email for myself to test forms, etc). I got the same "status 11" as I get from the Google servers, and my client's server is not blacklisted: (bb.bb.bb.bb is my client's server)

Code:

2019-09-27 00:04:20.877671500 tcpserver: status: 1/30
2019-09-27 00:04:20.877925500 tcpserver: pid 61084 from bb.bb.bb.bb
2019-09-27 00:04:20.900067500 tcpserver: ok 61084 pulp7.mydomain.com:192.168.0.154:25 :bb.bb.bb.bb::46606
2019-09-27 00:04:20.995035500 tcpserver: end 61084 status 11
2019-09-27 00:04:20.995037500 tcpserver: status: 0/30

What can I try next?

thanks

VladiBG · Sep 27, 2019

Use postfix.

Rob215x · Sep 27, 2019

Okay, I did a few more experiments. I used Thunderbird at home to try sending an email, using the custom port nnnnn that forwards to port 25 on my test server...

EXPERIMENT 1:
set Thunderbird SMTP to: custom port, Authentication: none

Sent email from postmaster@pulp7.mydomain.com TO postmaster@pulp7.domain.com:

qmail-smtpd log:

Code:

2019-09-27 02:37:29.895453500 tcpserver: status: 1/30
2019-09-27 02:37:29.895857500 tcpserver: pid 8785 from xx.xx.xx.xx
2019-09-27 02:37:29.915408500 tcpserver: ok 8785 pulp7.mydomain.com:192.168.0.154:25 c-xx-xx-xx-xx.hsd1.fl.comcast.net:xx.xx.xx.xx::39776
2019-09-27 02:37:29.954362500 qmail-smtpd[8785]: MAIL FROM:<postmaster@pulp7.mydomain.com>
2019-09-27 02:37:29.973385500 qmail-smtpd[8785]: RCPT TO:<postmaster@pulp7.mydomain.com>
2019-09-27 02:37:29.973387500 qmail-smtpd[8785]: validrcptto [xx.xx.xx.xx] trying: postmaster@pulp7.mydomain.com
2019-09-27 02:37:29.973388500 qmail-smtpd[8785]: validrcptto [xx.xx.xx.xx] found: postmaster@pulp7.mydomain.com
2019-09-27 02:37:30.071813500 tcpserver: end 8785 status 0
2019-09-27 02:37:30.071815500 tcpserver: status: 0/30

The message appeared in my Thunderbird INBOX immediately.

EXPERIMENT 2:
set Thunderbird SMTP to: custom port, Authentication: STARTTLS, normal password

Sent email from postmaster@pulp7.mydomain.com TO postmaster@pulp7.mydomain.com:

Thunderbird replied,
"Sending of the message failed:
The message could not be sent because the connection to Outgoing server (SMTP)
pulp7.mydomain.com was lost in the middle of the transaction. Try again."

qmail-smtpd log:

Code:

2019-09-27 02:45:39.419680500 tcpserver: status: 1/30
2019-09-27 02:45:39.420031500 tcpserver: pid 14334 from xx.xx.xx.xx
2019-09-27 02:45:39.421631500 tcpserver: ok 14334 pulp7.mydomain.com:192.168.0.154:25 c-xx-xx-xx-xx.hsd1.fl.comcast.net:xx.xx.xx.xx::39788
2019-09-27 02:45:39.514245500 tcpserver: end 14334 status 11
2019-09-27 02:45:39.514247500 tcpserver: status: 0/30

There's that status 11 again

EXPERIMENT 3:
set Thunderbird SMTP to: custom port, Authentication: SSL/TLS, normal password

Sent email from postmaster@pulp7.mydomain.com TO postmaster@pulp7.mydomain.com:

Thunderbird hung with a progress bar for a while...

The qmail-smtpd log hung like this:

Code:

2019-09-27 02:51:18.516484500 tcpserver: status: 1/30
2019-09-27 02:51:18.516708500 tcpserver: pid 15852 from xx.xx.xx.xx
2019-09-27 02:51:18.518182500 tcpserver: ok 15852 pulp7.mydomain.com:192.168.0.154:25 c-xx-xx-xx-xx.hsd1.fl.comcast.net:xx.xx.xx.xx::39798

After 40 seconds Thunderbird timed out with the message:
"Sending of the message failed:
The message could not be sent because the connection to Outgoing server (SMTP)
pulp7.mydomain.com timed out. Try again."

at the same time, the log updated:

Code:

2019-09-27 02:52:58.647813500 tcpserver: end 15852 status 256
2019-09-27 02:52:58.647816500 tcpserver: status: 0/30

Any ideas what all of this means? Qmail doesn't like STARTTLS on port 25?

VladiBG · Sep 27, 2019

Your running script is not including certificates for SSL or TLS that's why you can't use 465 or TLS on 25.

But because this is your new test server my advice is to start learning postfix instead of qmail. Qmail requires too many patches to add the functionality.

toorski · Sep 27, 2019

As mentioned above, Qmail would not be my choice for SMTP/MTA email server.
Moreover, no matter which SMTP/MTA server you use, here are some basics:

How email works: step-by-step, diagram

As simple as it is to use, an email relies on a more complicated set of operating procedures than that of the Web. For most users, its operation is transparent, which means that it is not necessary to understand how it works in order to be able to us...

ccm.net

Internet SMTP/MTA email servers EXCHANGE email messages with each other over port 25, without users authorization or authentication mechanisms, SSL or TLS!

It’s similar to that of FQDNS (authoritative, masters and slaves) servers that exchange information over port 53.

The email client (end-user) should communicate with its SMTP/MTA server, to send email to the outside world, with some type of authorization/authentication mechanism (such as PAM from a local system) using SSL or TLS over standard port 587 or whatever custom port that you may choose, as long as the email client is aware of it.

Then, the MDA/POP or IMAP server, (ea. Dovecot) is responsible for retrieving and delivering email messages to users email clients from system’s mail dir, also with some type of authorization/authentication mechanism over standard ports such as 110 for POP, 143 or 993 for IMAP using SSL or TLS.

I use Open-SMTPD and Exim MTA/SMTP servers which are easy to understand and configure and work very well for me. Tho, Open-SMTPD supports only openssl 1.0* versions, as of this writing, so I’m using it in 11.3 jail. Exim is all good in 12.0 jail with 1.1* openssl.

Rob215x · Oct 1, 2019

Okay, I solved my issue. I stopped all Qmail services, then rebuilt/reinstalled Qmail using instructions from:

Qmail Preinstall Checklist

I now have Qmail working with SMTP-SSL (Port 465) and SMTP-TLS (Port 587).
- My previous messages sent from Gmail arrived shortly after I restarted all of the services.
- I only followed his instructions up to and including Dovecot.
- I have not installed Rainloop (I don't need webmail), or the other things on his site that come after Dovecot.
- I still have no idea why my original Qmail install wasn't working, but sometimes I guess a fresh reinstall is necessary.

Thanks for everyone's suggestions

Solved Qmail will not receive mail from outside

Administrator