Trying to forward root to my GMail account

Hi all,

I originally posted this in a different thread in the General forum, but this has stopped being about general FreeBSD issues and started being specifically about sendmail, so I thought I would move it to what looks like the correct subforum.

I have FreeBSD installed on my fileserver. This isn't a machine that I log into normally on the command line unless something is wrong. However, several things send emails to the root account that I would like to get.

A friendly fellow by the name of Jose suggested I simply alias root to my personal address (in this case, a GMail account) in /etc/aliases, run newaliases, and then send a message from the command line to test that it worked. And for a bit it did. Then I ran into several other issues (all of which ended up being local networking configurations) and it stopped working until I fixed those. Then it worked again.

Fast forward to today, it's stopped working again. Apparently GMail has blocked me; I get this from their server (I've obfuscated all of the personal details):

Code:
yourname@gmail.com... Connecting to gmail-smtp-in.l.google.com. via esmtp...
220 mx.google.com ESMTP f37si6243403qte.211 - gsmtp
>>> EHLO fileserver.mysubdomain.mydomain.net
250-mx.google.com at your service, [136.56.39.40]
250-SIZE 157286400
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
>>> STARTTLS
220 2.0.0 Ready to start TLS
>>> EHLO fileserver.mysubdomain.mydomain.net
250-mx.google.com at your service, [136.56.39.40]
250-SIZE 157286400
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
>>> MAIL From:<root@fileserver.mysubdomain.mydomain.net> SIZE=69
250 2.1.0 OK f37si6243403qte.211 - gsmtp
>>> RCPT To:<yourname@gmail.com>
>>> DATA
250 2.1.5 OK f37si6243403qte.211 - gsmtp
354  Go ahead f37si6243403qte.211 - gsmtp
>>> .
550-5.7.1 [X.X.X.X] The IP you're using to send mail is not authorized to
550-5.7.1 send email directly to our servers. Please use the SMTP relay at your
550-5.7.1 service provider instead. Learn more at
550 5.7.1  https://support.google.com/mail/?p=NotAuthorizedError f37si6243403qte.211 - gsmtp
root... aliased to yourname@gmail.com
/root/dead.letter... Saved message in /root/dead.letter
Closing connection to gmail-smtp-in.l.google.com.
>>> QUIT

Unfortunately, that's not helpful, because Google Fiber is my service provider, and they don't seem to have an SMTP relay.

I actually have my domain via Google Domains, too, but if there's any way to change the DNS records to make this work I don't know what it is. My outward facing IP is dynamic, and while forward lookup works, reverse lookup doesn't work. The FreeBSD fileserver isn't connected directly to the internet anyway; even if reverse lookup did work it would return mygateway.mydomain.net instead of fileserver.mysubdomain.mydomain.net.

I don't actually know if any of that matters anyway, because while I have a semi-competent grasp on DNS I don't understand email at all.

Anyone have any suggestions of things to try or troubleshooting avenues to go down? I'm at a complete loss here, I don't even know where to start. The only thing I'm certain of is that the local mail alias is still working.
 
I would open a ticket with Google Fiber with these details.

The roll-your-own solution is not going to be simple. You'll need to set up a Mail Transfer Agent (MTA) on a machine with a static IP. You'll then have to have your server at your house authenticate with that machine and use it to send mail. That might be enough, but you might also have to set up MX and SPF records in DNS for your public server. You'll have to protect your public server 'cause every spammer on the Internet is going to start trying to crack your authentication to use you to send SPAM.

I've been working on an Ansible script to automate the setup of a mail server using a Digital Ocean Freebsd droplet, but it's not ready for public consumption yet.
 
You don't have to have a static IP to contact Google's SMTP servers to forward to your gmail account. But:
  • you will have to identify yourself with your gmail/gsuite account (userid and password required);
  • you do have to use an X509 certificate;
  • the header "From:" line must end with "@gmail.com"; and
  • the sending agent (you can masquerade) must have an Internet DNS record.
I masquerade, using the same identity, from several different hosts, including my notebook when on the road.

I didn't record what types of Internet DNS records are required, but I have MX, A, and reverse lookups for the masqueraded identity, which is my domain name.

When I set mine up, I used sendmail (had to be compiled from ports with -DSASL), and broadly followed the instructions in the posting by "granth" here.

I kept detailed records of what I did for my FreeBSD hosts, and I'm happy to share them if you want. But it's sendmail(8) specific.
 
Perhaps the easiest way to solve it is configuring sendmail to use your google or other account as SMART_HOST.

But remember: when you send an email to yourself in google, it does not land in the INBOX folder, perhaps you will need to use a filter.

The other problem is, that google authentication is getting always more difficult. Perhaps is better to use other mail account.

Other simple solution: configure a simple mail program like mail/heirloom-mailx to send mails with google and write a script using it that is started with .forward or /etc/aliases mechanism.

Note that your problem is that you are not using gmail for sending mail, but for relaying mail, and I doubt that google has that service.
 
I kept detailed records of what I did for my FreeBSD hosts, and I'm happy to share them if you want. But it's sendmail(8) specific.
I think this would be worthwhile, since this is likely a very common use case. The default mailer in the base system is Sendmail, and most folks are unlikely to want to set up a full-blown MTA.
 
Here is the README.Google file from /etc/mail
Code:
My sendmail is configured to forward through Google's SMTP servers.

This documents what I did to make it work.  It was done a few years ago,
but continues to work for me.

I actually use G Suite (which means that the MX record for my domain points
to Google's SMTP servers, and I pay them for a mail server service).
It costs me Aus$100 per year to have Google to deal with inbound SPAM,
and I don't have to have port 25 open to the Internet.  Bargain...
I document the Gmail setup here, and the small changes required for G Suite
are at the end.

I use sendmail, and this is a guide for sendmail.  I started by broadly
following the instructions in the posting by "granth" here:

  https://www.linuxquestions.org/questions/slackware-14/how-to-configure-sendmail-to-use-gmail-as-smtp-server-802815/

You need to be root throughout, and data field commencing "My" is not to be
taken literally, and needs to be changed to suit your identity.

################################################################################

1.  Get sendmail installed with the -DSASL option

I think it might have it by default these days, but originally I had
to build and install the mail/sendmail port, specifically enabling SASL.

You may have to change the definition of sendmail in /etc/mail/mailer.conf:

    #
    # Execute the "real" sendmail program, named /usr/libexec/sendmail/sendmail
    #
    sendmail    /usr/libexec/sendmail/sendmail
    mailq    /usr/libexec/sendmail/sendmail
    newaliases    /usr/libexec/sendmail/sendmail
    hoststat    /usr/libexec/sendmail/sendmail
    purgestat    /usr/libexec/sendmail/sendmail


################################################################################

2.  Create your SSL certs

mkdir /etc/mail/certs
chmod 700 /etc/mail/certs
cd /etc/mail/certs
openssl req -new -x509 -keyout mykey.pem -out mycert.pem -days 3650

################################################################################

3.  Set up your Gmail login credentials 

mkdir -p /etc/mail/authinfo
chmod 600 /etc/mail/authinfo
cd /etc/mail/authinfo
echo 'AuthInfo: "U:root" "I:My.Name@gmail.com" "MyGmailPassword"' >gmail.auth
makemap -r hash gmail.auth.db < gmail.auth
chmod 600 gmail.*    # this may trip you up later, so remember it for mow.

################################################################################

4.  Modify the default sendmail configuration

cd /etc/mail
cp freebsd.mc $(hostname).mc

Edit $(hostname).mc to make the following changes:

diff freebsd.mc $(hostname).mc
54a55,59
> dnl To use the binaries supplied by the port you should add the following lines
> dnl to your sendmail.mc file before any mailer or feature definition:
> define(`confEBINDIR', `/usr/local/libexec')dnl
> define(`UUCP_MAILER_PATH', `/usr/local/bin/uux')dnl
> 
60a66,74
> dnl FEATURE(`genericstable')dnl
> FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail.auth.db')dnl
> 
> dnl Google's SMTP servers need to do a DNS lookup on us (sending agent)
> MASQUERADE_AS(My.Domain)dnl
> FEATURE(masquerade_envelope)dnl
> FEATURE(masquerade_entire_domain)dnl
> MASQUERADE_DOMAIN(My.Domain)dnl
> 
63,67d76
< define(`confSERVER_CERT', `CERT_DIR/host.cert')dnl
< define(`confSERVER_KEY', `CERT_DIR/host.key')dnl
< define(`confCLIENT_CERT', `CERT_DIR/host.cert')dnl
< define(`confCLIENT_KEY', `CERT_DIR/host.key')dnl
< define(`confCACERT', `CERT_DIR/cacert.pem')dnl
69c78,83
< define(`confDH_PARAMETERS', `CERT_DIR/dh.param')dnl
---
> define(`confCACERT', `CERT_DIR/mycert.pem')dnl
> define(`confSERVER_CERT', `CERT_DIR/mycert.pem')dnl
> define(`confSERVER_KEY', `CERT_DIR/mykey.pem')dnl
> define(`confCLIENT_CERT', `CERT_DIR/mycert.pem')dnl
> define(`confCLIENT_KEY', `CERT_DIR/mykey.pem')dnl
> dnl WTF define(`confDH_PARAMETERS', `CERT_DIR/dh.param')dnl
98c112,114
< DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')
---
> dnl DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')
> 
> define(`confMAX_HOP', 40)
104a121,127
> 
> define(`SMART_HOST',`[smtp.gmail.com]')dnl
> define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
> define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
> define(`confAUTH_OPTIONS', `A p')dnl
> TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
> define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl

################################################################################

5.  Make and install the new sendmail.cf

cd /etc/mail
make install
service sendmail restart

################################################################################

6. Comply with Google's rules:

Google's SMTP servers require:

  - the header "From:" line ends with "@gmail.com"; and
  - the sending agent has in Internet DNS record.

I fixed (1) with $HOME/.elm/elmheaders:

    From: "My Name" <MyName@gmail.com>
    Reply-To: "My Name" <MyName@gmail.com>
    Errors-To: "My Name" <MyName@gmail.com>

I fixed (2) with masquerading (My.Domain is registered, and the 
provider [NameSilo] publishes free Internet visible MX, A, and inaddr.arpa
DNS records).  These changes are shown above in the ".mc" file:

    dnl Google's SMTP servers need to do a DNS lookup
    MASQUERADE_AS(My.Domain)dnl
    FEATURE(masquerade_envelope)dnl
    FEATURE(masquerade_entire_domain)dnl
    MASQUERADE_DOMAIN(My.Domain)dnl

################################################################################

I use fetchmail (POP3) to get my mail from Google.

Others do it differently.  e.g. you can:

  - sync gmail mail/isync.  See $HOME/.mbsyncrc (which works); and
  - send with mail/ssmtp.  See /usr/local/etc/ssmtp/ssmtp.conf (untested).

I stuck with fetchmail, because isync mirrors gmail's folder structures
(with one email per file) and I prefer a single file mail box which is
integrated with procmail (well, I think I do, and changing is just more
work...).

The .fetchmailrc looks like this:

    # The sslfingerprints (in $HOME/.fetchmailrc) come from:
    # echo -n | openssl s_client -connect pop.gmail.com:995 2>/dev/null | \
    #   sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | \
    #   openssl x509 -noout -fingerprint -md5 | cut -d '=' -f 2

    set logfile "/usr/home/MyLoginName/.fetchmail/fetchmail.log"
    set idfile "/usr/home/MyLoginName/.fetchmail/fetchids"
    set postmaster "MyLoginName"
    set no bouncemail
    set properties ""
    poll pop.gmail.com
      with no dns
      with proto POP3
      user "MyName@gmail.com"
      pass "MyGmailPassword"
      is MyLoginName here 
    options
      warnings 3600
      antispam 571 550 501 554
      no keep
      ssl
      sslcertck
      sslcertfile /usr/local/share/certs/ca-root-nss.crt

################################################################################

But setting up your Unix mail server for G Suite (My.Chosen.Name@My.Domain)
is very similar to Gmail (My.Name@gmail.com).  Once you have Gmail working,
G Suite is very similar to Gmail, except:

  - sign up for G Suite, and do some setup for your domain with Google GUIs;
  - generally s/gmail/gsuite/ throughout;
  - you have to authenticate as your.name@your.domain (in ./authinfo);
  - your mail headers have to reflect the same (e.g. in $HOME/.elm/elmheaders);
  - you can use a different SMART_HOST iff you have a fixed IP Address.

################################################################################
 
BTW. perhaps he should from time to time click on "allow less secure apps" in the page of his google account.
That's certainly true, thanks. I did the Google setup via their GUI several years ago, and didn't keep notes...
 
the provider [NameSilo] publishes free Internet visible MX, A, and inaddr.arpa DNS records).
I should add that I don't have a permanent IPv4 address.
The the A record for your masqueraded "From:" host (My.Domain) needs to exist. It's OK if it points to a place holder which providers generally set up as the default for "squatting" (in my case My.Domain resolves 107.161.23.204).
I don't know if the reverse lookup is required. Mine does. 107.161.23.204 resolves to "parking.namesilo.com". So my Internet A record and reverse lookup is a pretty useless "squat", but it exists (and I can change it at any time).
If you use a dynamic DNS service and want reverse lookups to work correctly, you have to get it all from the provider of your dynamic IP address (because their DNS servers own the entire block of IP addresses and are the only servers that can publish the inarpa.addr records for that block).
If your dial-up service provider does not provide a Dynamic DNS service for the IP address they allocate to you when you dial in, and you want reverse lookups to work, you will have to acquire a permanent IPv4 address.
If you use G Suite, the MX record for My.Domain needs to point to Google's SMTP servers . You don't need it otherwise.
 
Okay, lots of good info here. Let me dig in to the individual responses...

I would open a ticket with Google Fiber with these details.

The roll-your-own solution is not going to be simple. You'll need to set up a Mail Transfer Agent (MTA) on a machine with a static IP. You'll then have to have your server at your house authenticate with that machine and use it to send mail. That might be enough, but you might also have to set up MX and SPF records in DNS for your public server. You'll have to protect your public server 'cause every spammer on the Internet is going to start trying to crack your authentication to use you to send SPAM.

I've been working on an Ansible script to automate the setup of a mail server using a Digital Ocean Freebsd droplet, but it's not ready for public consumption yet.

Unfortunately this is not actually doable for me. I don't have access to a single machine that has a static public IP, and even if I did I really don't want the responsibility of running my own mailserver. In fact I don't want to run anything for public use, even my local caching resolver (which is authoritative only for my internal subdomains) isn't accessible from the outside world.

You don't have to have a static IP to contact Google's SMTP servers to forward to your gmail account. But:
  • you will have to identify yourself with your gmail/gsuite account (userid and password required);
  • you do have to use an X509 certificate;
  • the header "From:" line must end with "@gmail.com"; and
  • the sending agent (you can masquerade) must have an Internet DNS record.
I masquerade, using the same identity, from several different hosts, including my notebook when on the road.

I didn't record what types of Internet DNS records are required, but I have MX, A, and reverse lookups for the masqueraded identity, which is my domain name.

When I set mine up, I used sendmail (had to be compiled from ports with -DSASL), and broadly followed the instructions in the posting by "granth" here.

I kept detailed records of what I did for my FreeBSD hosts, and I'm happy to share them if you want. But it's sendmail(8) specific.

I don't know how to do any of this, but I'm happy to try. I'd need instructions on how to do all of this though.

I have my own domain I can masquerade as, but I'm not sure which host to point the "MX" record at--do I just point it "gateway.mydomain.com", the only hostname I have that has a publicly routable IP address? And I can't seem to get a reverse lookup...despite having Google Fiber as my ISP, Google's own DNS servers respond NXDOMAIN when I try to lookup my IP address. Google Domains supports PTR records, but I can't figure out how one would point to the dynamic IP with them (I know how to setup reverse lookup, my local DNS server provides it for all my internal hosts--but those are static, RFC1918 private addresses). I do already have the "A" record and it works, that is the address I use to connect in from remote.

Perhaps the easiest way to solve it is configuring sendmail to use your google or other account as SMART_HOST.

But remember: when you send an email to yourself in google, it does not land in the INBOX folder, perhaps you will need to use a filter.

The other problem is, that google authentication is getting always more difficult. Perhaps is better to use other mail account.

Other simple solution: configure a simple mail program like mail/heirloom-mailx to send mails with google and write a script using it that is started with .forward or /etc/aliases mechanism.

Note that your problem is that you are not using gmail for sending mail, but for relaying mail, and I doubt that google has that service.

I don't actually know what a SMART_HOST is. Can you explain?

I don't have any other mail account that I actually read--I just don't have the time for that. I have some I use for SPAM, but I literally never check them. I only check my work account and my personal GMail, and for obvious reasons I don't want to start bouncing my personal network status emails off of my work servers/mail account.

I had this working briefly, I had a couple of emails arrive in GMail (they went straight to Spam, so yes, I had to use a filter). Every single one of them was sent before I had to split my network into two subdomains, though. Perhaps the solution is that I just need my emails to go back to looking like they're coming from "mydomain.net" instead of "subdomain.mydomain.net"? But I don't know how to do this--sendmail is incredibly opaque, much worse than BIND IMO.

I've already set up less secure apps in GMail. I use Thunderbird for reading mail and that's been required for a long time now. If that had gotten turned off, Thunderbird would have stopped working. Just to be sure, I doublechecked, and it is still on (less secure app access is allowed).

Here is the README.Google file from /etc/mail
Code:
My sendmail is configured to forward through Google's SMTP servers.

This documents what I did to make it work.  It was done a few years ago,
but continues to work for me.

I actually use G Suite (which means that the MX record for my domain points
to Google's SMTP servers, and I pay them for a mail server service).
It costs me Aus$100 per year to have Google to deal with inbound SPAM,
and I don't have to have port 25 open to the Internet.  Bargain...
I document the Gmail setup here, and the small changes required for G Suite
are at the end.

I use sendmail, and this is a guide for sendmail.  I started by broadly
following the instructions in the posting by "granth" here:

  https://www.linuxquestions.org/questions/slackware-14/how-to-configure-sendmail-to-use-gmail-as-smtp-server-802815/

You need to be root throughout, and data field commencing "My" is not to be
taken literally, and needs to be changed to suit your identity.

################################################################################

1.  Get sendmail installed with the -DSASL option

I think it might have it by default these days, but originally I had
to build and install the mail/sendmail port, specifically enabling SASL.

You may have to change the definition of sendmail in /etc/mail/mailer.conf:

    #
    # Execute the "real" sendmail program, named /usr/libexec/sendmail/sendmail
    #
    sendmail    /usr/libexec/sendmail/sendmail
    mailq    /usr/libexec/sendmail/sendmail
    newaliases    /usr/libexec/sendmail/sendmail
    hoststat    /usr/libexec/sendmail/sendmail
    purgestat    /usr/libexec/sendmail/sendmail


################################################################################

2.  Create your SSL certs

mkdir /etc/mail/certs
chmod 700 /etc/mail/certs
cd /etc/mail/certs
openssl req -new -x509 -keyout mykey.pem -out mycert.pem -days 3650

################################################################################

3.  Set up your Gmail login credentials

mkdir -p /etc/mail/authinfo
chmod 600 /etc/mail/authinfo
cd /etc/mail/authinfo
echo 'AuthInfo: "U:root" "I:My.Name@gmail.com" "MyGmailPassword"' >gmail.auth
makemap -r hash gmail.auth.db < gmail.auth
chmod 600 gmail.*    # this may trip you up later, so remember it for mow.

################################################################################

4.  Modify the default sendmail configuration

cd /etc/mail
cp freebsd.mc $(hostname).mc

Edit $(hostname).mc to make the following changes:

diff freebsd.mc $(hostname).mc
54a55,59
> dnl To use the binaries supplied by the port you should add the following lines
> dnl to your sendmail.mc file before any mailer or feature definition:
> define(`confEBINDIR', `/usr/local/libexec')dnl
> define(`UUCP_MAILER_PATH', `/usr/local/bin/uux')dnl
>
60a66,74
> dnl FEATURE(`genericstable')dnl
> FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail.auth.db')dnl
>
> dnl Google's SMTP servers need to do a DNS lookup on us (sending agent)
> MASQUERADE_AS(My.Domain)dnl
> FEATURE(masquerade_envelope)dnl
> FEATURE(masquerade_entire_domain)dnl
> MASQUERADE_DOMAIN(My.Domain)dnl
>
63,67d76
< define(`confSERVER_CERT', `CERT_DIR/host.cert')dnl
< define(`confSERVER_KEY', `CERT_DIR/host.key')dnl
< define(`confCLIENT_CERT', `CERT_DIR/host.cert')dnl
< define(`confCLIENT_KEY', `CERT_DIR/host.key')dnl
< define(`confCACERT', `CERT_DIR/cacert.pem')dnl
69c78,83
< define(`confDH_PARAMETERS', `CERT_DIR/dh.param')dnl
---
> define(`confCACERT', `CERT_DIR/mycert.pem')dnl
> define(`confSERVER_CERT', `CERT_DIR/mycert.pem')dnl
> define(`confSERVER_KEY', `CERT_DIR/mykey.pem')dnl
> define(`confCLIENT_CERT', `CERT_DIR/mycert.pem')dnl
> define(`confCLIENT_KEY', `CERT_DIR/mykey.pem')dnl
> dnl WTF define(`confDH_PARAMETERS', `CERT_DIR/dh.param')dnl
98c112,114
< DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')
---
> dnl DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')
>
> define(`confMAX_HOP', 40)
104a121,127
>
> define(`SMART_HOST',`[smtp.gmail.com]')dnl
> define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
> define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
> define(`confAUTH_OPTIONS', `A p')dnl
> TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
> define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl

################################################################################

5.  Make and install the new sendmail.cf

cd /etc/mail
make install
service sendmail restart

################################################################################

6. Comply with Google's rules:

Google's SMTP servers require:

  - the header "From:" line ends with "@gmail.com"; and
  - the sending agent has in Internet DNS record.

I fixed (1) with $HOME/.elm/elmheaders:

    From: "My Name" <MyName@gmail.com>
    Reply-To: "My Name" <MyName@gmail.com>
    Errors-To: "My Name" <MyName@gmail.com>

I fixed (2) with masquerading (My.Domain is registered, and the
provider [NameSilo] publishes free Internet visible MX, A, and inaddr.arpa
DNS records).  These changes are shown above in the ".mc" file:

    dnl Google's SMTP servers need to do a DNS lookup
    MASQUERADE_AS(My.Domain)dnl
    FEATURE(masquerade_envelope)dnl
    FEATURE(masquerade_entire_domain)dnl
    MASQUERADE_DOMAIN(My.Domain)dnl

################################################################################

I use fetchmail (POP3) to get my mail from Google.

Others do it differently.  e.g. you can:

  - sync gmail mail/isync.  See $HOME/.mbsyncrc (which works); and
  - send with mail/ssmtp.  See /usr/local/etc/ssmtp/ssmtp.conf (untested).

I stuck with fetchmail, because isync mirrors gmail's folder structures
(with one email per file) and I prefer a single file mail box which is
integrated with procmail (well, I think I do, and changing is just more
work...).

The .fetchmailrc looks like this:

    # The sslfingerprints (in $HOME/.fetchmailrc) come from:
    # echo -n | openssl s_client -connect pop.gmail.com:995 2>/dev/null | \
    #   sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | \
    #   openssl x509 -noout -fingerprint -md5 | cut -d '=' -f 2

    set logfile "/usr/home/MyLoginName/.fetchmail/fetchmail.log"
    set idfile "/usr/home/MyLoginName/.fetchmail/fetchids"
    set postmaster "MyLoginName"
    set no bouncemail
    set properties ""
    poll pop.gmail.com
      with no dns
      with proto POP3
      user "MyName@gmail.com"
      pass "MyGmailPassword"
      is MyLoginName here
    options
      warnings 3600
      antispam 571 550 501 554
      no keep
      ssl
      sslcertck
      sslcertfile /usr/local/share/certs/ca-root-nss.crt

################################################################################

But setting up your Unix mail server for G Suite (My.Chosen.Name@My.Domain)
is very similar to Gmail (My.Name@gmail.com).  Once you have Gmail working,
G Suite is very similar to Gmail, except:

  - sign up for G Suite, and do some setup for your domain with Google GUIs;
  - generally s/gmail/gsuite/ throughout;
  - you have to authenticate as your.name@your.domain (in ./authinfo);
  - your mail headers have to reflect the same (e.g. in $HOME/.elm/elmheaders);
  - you can use a different SMART_HOST iff you have a fixed IP Address.

################################################################################

Large parts of this look like they're intended primarily for fetching, reading, and sending GMail on your FreeBSD machine as a local user, which I don't have any need for. The only thing I need is for sendmail to be able to forward mail arriving at root's local account to my personal GMail. Can you tell me which parts of this are actually necessary for this? I don't use GSuite and am not sending through Elm or Pine or any other program designed to send mail as a specific user (brings back memories, though, I remember using PINE for mail in the mid 90s).

I do have functioning Dynamic DNS on my domain (at least the "A" record); anyone can lookup "gateway.mydomain.net" and have it successfully resolve to my external IP, and I have ddclient on gateway keeping that updated. Reverse lookup ("PTR" record) doesn't seem to work, though, and I can't seem to figure out anyway to make it work.

Edit: Also, for step 2, a bunch of files already exist in /etc/mail/certs, and I'm not sure what I should do with this:

Code:
fileserver:/etc/mail/certs# ls
total 14
lrwxr-xr-x  1 root  wheel    10 May 13 15:27 c76e12ea.0@ -> cacert.pem
-rw-r--r--  1 root  wheel  1338 May 13 15:27 cacert.pem
-rw-r--r--  1 root  wheel  1371 May 13 15:27 host.cert
-rw-------  1 root  wheel  1708 May 13 15:27 host.key
 
I did set up the MX record, though I have no idea if it works right; I've never set one up before:

Code:
fileserver:/root# dig mydomain.net MX

; <<>> DiG 9.16.2 <<>> mydomain.net MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14102
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 9

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 280bcf419afc1fbca06898215ed3e6eeeaed2ddbe9fb44df (good)
;; QUESTION SECTION:
;mydomain.net.                  IN      MX

;; ANSWER SECTION:
mydomain.net.           3479    IN      MX      10 mail.mydomain.net.

;; AUTHORITY SECTION:
mydomain.net.           21479   IN      NS      ns-cloud-b1.googledomains.com.
mydomain.net.           21479   IN      NS      ns-cloud-b4.googledomains.com.
mydomain.net.           21479   IN      NS      ns-cloud-b2.googledomains.com.
mydomain.net.           21479   IN      NS      ns-cloud-b3.googledomains.com.

;; ADDITIONAL SECTION:
ns-cloud-b1.googledomains.com. 30961 IN A       216.239.32.107
ns-cloud-b2.googledomains.com. 30961 IN A       216.239.34.107
ns-cloud-b3.googledomains.com. 30961 IN A       216.239.36.107
ns-cloud-b4.googledomains.com. 30961 IN A       216.239.38.107
ns-cloud-b1.googledomains.com. 30961 IN AAAA    2001:4860:4802:32::6b
ns-cloud-b2.googledomains.com. 30961 IN AAAA    2001:4860:4802:34::6b
ns-cloud-b3.googledomains.com. 30961 IN AAAA    2001:4860:4802:36::6b
ns-cloud-b4.googledomains.com. 30961 IN AAAA    2001:4860:4802:38::6b

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sun May 31 13:18:38 EDT 2020
;; MSG SIZE  rcvd: 387

"mail.mydomain.net" is then a CNAME that points to "gateway.mydomain.net", and I can verify that works, at least:

Code:
fileserver:/root# ping mail.mydomain.net
PING gateway.mydomain.net (136.56.39.40): 56 data bytes
64 bytes from X.X.X.X: icmp_seq=0 ttl=255 time=0.275 ms
64 bytes from X.X.X.X: icmp_seq=1 ttl=255 time=0.306 ms
64 bytes from X.X.X.X: icmp_seq=2 ttl=255 time=0.304 ms
64 bytes from X.X.X.X: icmp_seq=3 ttl=255 time=0.249 ms
^C
--- gateway.mydomain.net ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.249/0.283/0.306/0.023 ms
 
I don't actually know what a SMART_HOST is. Can you explain?

As I see, the configuration of gpw928 is as SMART_HOST. You may google SMART_HOST+sendmail, there is sure a lot of descriptions and instructions. It is about configuring sendmail so, that mails send by your system will look like as they were sent from (not to) your mail account (google or other).

If you use an account other than from google, you do not need to read it: from that account mails will be sent to your google account through the alias mechanism. You will only need to read mails in your google account. You can also configure the new account to forward all mail to your google account in case that mails are sent to that account, but that is not the case here.

On the meantime, google is continuously disabling "less secure apps". It will perhaps come the time, at which it will be impossible to read gmail with other app than a google app, namely to use a normal mail client or sendmail as SMART_HOST with gmail.
 
I have my own domain I can masquerade as, but I'm not sure which host to point the "MX" record at...
You shouldn't need an MX record at all. They're mainly used for receiving email. They're also needed to set up an SPF record, but it looks like Google does not require them.

...And I can't seem to get a reverse lookup...
Looks like that's not required either.

I don't actually know what a SMART_HOST is. Can you explain?
A smart host is just a properly-configured machine running Mail Transfer Agent (MTA) software that authenticates you and relays your messages to their final destination. Because SPAM there are lots of complications involved in configuring an MTA properly. See here for example: http://rfc-clueless.org/pages/listing_policy-bogusmx

In your case, you're going to authenticate with Google's smart hosts, and they're going to forward mail for you.

I had this working briefly, I had a couple of emails arrive in GMail (they went straight to Spam, so yes, I had to use a filter). Every single one of them was sent before I had to split my network into two subdomains, though. Perhaps the solution is that I just need my emails to go back to looking like they're coming from "mydomain.net" instead of "subdomain.mydomain.net"? But I don't know how to do this--sendmail is incredibly opaque, much worse than BIND IMO.
This is worth a try. You can masquerade as mydomain.net without the subdomain. Gpw928's instructions cover configuring masquerading with Sendmail in step 6. Mjollnir recommended dma(8) in your other thread. I've never used it, but I get the impression that it's much, much easier to configure than Sendmail from reading its man page.

Large parts of this look like they're intended primarily for fetching, reading, and sending GMail on your FreeBSD machine as a local user, which I don't have any need for. The only thing I need is for sendmail to be able to forward mail arriving at root's local account to my personal GMail. Can you tell me which parts of this are actually necessary for this?
I believe you can ignore everything from "I use fetchmail (POP3) to get my mail from Google..." to the end.
 
A smart host is just a properly-configured machine running Mail Transfer Agent (MTA) software that authenticates you and relays your messages to their final destination. Because SPAM there are lots of complications involved in configuring an MTA properly.

That definition means "relay with authentication". SMART_HOST is more than that, it is using your MTA as a mail client for other MTA. You may see here:


And for using a mail client, even if it is your MTA program, you do not need domains, MX records, reverse lookups, SPF, DKIM, static IP, etc.
 
I did my best to follow gpw928's instructions. I don't know if I followed it correctly; specifically the diff for $(hostname).mc was really hard for me to follow with no context lines in the diff as the line numbers didn't seem to match up with what was in freebsd.mc. I did have to compile the ports version of sendmail to get SASL support, as it complained when I did the make install. However, despite having access for less secure apps turned on, I get this:

Code:
yourname@gmail.com... Connecting to smtp.gmail.com. port 587 via relay...
220 smtp.gmail.com ESMTP p25sm14157920qtj.18 - gsmtp
>>> EHLO fileserver.subdomain.mydomain.net
250-smtp.gmail.com at your service, [136.56.39.40]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
>>> STARTTLS
220 2.0.0 Ready to start TLS
>>> EHLO fileserver.subdomain.mydomain.net
250-smtp.gmail.com at your service, [136.56.39.40]
250-SIZE 35882577
250-8BITMIME
250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
>>> MAIL From:<root@fileserver.subdomain.mydomain.net> SIZE=69
530-5.7.0 Authentication Required. Learn more at
530 5.7.0  https://support.google.com/mail/?p=WantAuthError p25sm14157920qtj.18 - gsmtp
root... aliased to yourname@gmail.com
/root/dead.letter... Saved message in /root/dead.letter
Closing connection to smtp.gmail.com.
>>> QUIT
221 2.0.0 closing connection p25sm14157920qtj.18 - gsmtp

I'm assuming the at least problem is the From: line (it should say yourname@gmail.com, I'm guessing), but I don't know how to fix it. The link provided was absolutely useless, just like every link Google has ever given me whenever I have a problem.
 
Looks like you're not authenticating with your username and password. That was set up in steps 3 and 4 of Gpw928's guide. Step 3 creates files with your login info. This part of step 4 FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail.auth.db')dnl configures Sendmail to use them (I think).
 
If you want a local ISP to be your SMART_HOST, and use an email address like "MyName@MyISPDomain" you may not need to do much other than set up the SMART_HOST, and maybe use sendmail's genericstable with an entry like "MyLoginName MyName@MyISPDomain". That all depends on your local ISP's requirements.

If your host does not have a permanent IP address, and you want to send from an email address like MyName@gmail.com and use Google's SMTP servers to be your SMART_HOST you have to do everything I suggested.

I also forgot to mention that you also need to register a TXT record (google-site-verification) for your (masqueraded) domain. But that's no more difficult than registering any other sort of DNS record with your DNS service provider (Google will provide the content).

Google simply won't accept any old email (SPAM management). The must know who you are, and you must identify yourself in the way that Google demands.

As others have observed, you don't need an MX record unless you wish to receive email to your own registered domain (which, in this context, is G Suite specific). An MX record for a domain points to an IP address that has a mail agent listening on port 25, and willing to accept mail messages addressed to that registered domain.

If you don't want to fetch mail from Google to read locally (as opposed to using a web browser pointed at Google), you can ignore the POP3 stuff. I put it there because it's in my README file, and for completion.

If you don't interactively compose email on your host, then ignore all the stuff about elm (it's there as an example). But you still have to address the "From:" issue when sending/forwarding. So use an agent that lets you easily change the "From:" line to test. They should all offer a way to do that. Elm was just the simplest example I could find.

The fact that all you want is "for sendmail to be able to forward mail arriving at root's local account" changes nothing at the core. You are still sending mail from a host without a permanent IP address to Google's SMTP servers. That's what my process documents.

I see there have been posts while I was composing this...
 
I do have functioning Dynamic DNS on my domain (at least the "A" record); anyone can lookup "gateway.mydomain.net" and have it successfully resolve to my external IP, and I have ddclient on gateway keeping that updated. Reverse lookup ("PTR" record) doesn't seem to work, though, and I can't seem to figure out anyway to make it work.
I have addressed this above, but it's getting cluttered. PTR records can only be published by the owner of the IP address block delegation. So any public DNS can publish your A record, but only the delegated owner of the IP address block can publish the PTR record matching the A record. So, if you want reverse lookups to work correctly you must get your dynamic IP address and your dynamic DNS service from the same provider. This can be difficult... I don't know if the PTR has to exist. I suspect not, because most people in your position can't get one.
Edit: Also, for step 2, a bunch of files already exist in /etc/mail/certs, and I'm not sure what I should do with this:
I ignored what was there, and did exactly what the instructions in the post indicate. This may have been a bit messy, but it worked.
 
I do technically get my dynamic IP and dynamic DNS service from the same provider, but knowing Google, "Google Fiber" and "Google Domains" are just far enough apart that they don't qualify. I'm mostly just surprised the Google Fiber doesn't have ANY reverse lookup for their address block; back in the days when I had Verizon or Time Warner they all had at least something generic for it (usually something like 111-111-110-103.vsync.verizon.net, but still, something). Doesn't instill me with confidence, but then again, very little Google has done in the last ~5 years has.

Anyway, I followed the process as well as I could based on my limited understanding of what's going on. Steps 1-3 I followed without too much trouble, though I also used "-nodes" on the command line the second time around because I didn't want a passphrase (I tried it both with and without the passphrase, got the same results both ways, so at least for this particular failure, I don't think the missing passphrase is the issue). I did have a minor issue with step 3, in that makemap refuses to work if you chmod 600 /etc/mail/authinfo first, but that was easy to work around (just changed it to 770, ran makemap, then changed it back to 600).

One thing that is confusing me is I'm not 100% sure who I'm supposed to be masquerading as, since the machine I'm working on is behind a NAT firewall. Its local FQDN is "fileserver.subdomain.mydomain.net", but the firewall is actually "gateway.mydomain.net" (it has a local FQDN as well, gateway.subdomain.mydomain.net, but that probably isn't relevant). In all cases I used the internal FQDN of "fileserver.subdomain.mydomain.net" (for the certificate as well as all configuration files). I hope that's correct.

I attempted to send mail directly using the command line but it seems as though sendmail is completely ignoring the -f and -F options, and I have no idea how to fix that. If I can't get this to work I probably have no prayer of getting the automatic emails to work. Here's the command line I attempted to use. If I put the -f option before the --, it says ">>> MAIL FROM:<> SIZE=1093", if I put it after, it ignores it and I still get ">>> MAIL From:<root@fileserver.subdomain.mydomain.net> SIZE=69".

Code:
/usr/sbin/sendmail -i -v -Am -- yourname@gmail.com -f 'yourname@gmail.com' <<END
Subject: Delivery test
To: yourname@gmail.com

Delivery test.
END

I had meant to post the contents of my $(hostname).mc but got called away before I could do so. As i said, with no context lines it was pretty difficult for me to figure out where to put things or what lines need to be removed. I copied from vi with line numbers on; the numbers aren't actually there of course but if there's something to change I thought it would be helpful:

Code:
  1 divert(-1)
  2 #
  3 # Copyright (c) 1983 Eric P. Allman
  4 # Copyright (c) 1988, 1993
  5 #       The Regents of the University of California.  All rights reserved.
  6 #
  7 # Redistribution and use in source and binary forms, with or without
  8 # modification, are permitted provided that the following conditions
  9 # are met:
 10 # 1. Redistributions of source code must retain the above copyright
 11 #    notice, this list of conditions and the following disclaimer.
 12 # 2. Redistributions in binary form must reproduce the above copyright
 13 #    notice, this list of conditions and the following disclaimer in the
 14 #    documentation and/or other materials provided with the distribution.
 15 # 3. All advertising materials mentioning features or use of this software
 16 #    must display the following acknowledgement:
 17 #       This product includes software developed by the University of
 18 #       California, Berkeley and its contributors.
 19 # 4. Neither the name of the University nor the names of its contributors
 20 #    may be used to endorse or promote products derived from this software
 21 #    without specific prior written permission.
 22 #
 23 # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 24 # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 25 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 26 # ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 27 # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 28 # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 29 # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 30 # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 31 # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 32 # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 33 # SUCH DAMAGE.
 34 #
 35
 36 #
 37 #  This is a generic configuration file for FreeBSD 6.X and later systems.
 38 #  If you want to customize it, copy it to a name appropriate for your
 39 #  environment and do the modifications there.
 40 #
 41 #  The best documentation for this .mc file is:
 42 #  /usr/share/sendmail/cf/README or
 43 #  /usr/src/contrib/sendmail/cf/README
 44 #
 45 #  NOTE: If you enable RunAsUser, make sure that you adjust the permissions
 46 #  and owner of the SSL certificates and keys in /etc/mail/certs to be usable
 47 #  by that user.
 48 #
 49
 50 divert(0)
 51 VERSIONID(`$FreeBSD: releng/12.1/etc/sendmail/freebsd.mc 285230 2015-07-07 03:00:57Z gshapiro $')
 52 OSTYPE(freebsd6)
 53 DOMAIN(generic)
 54
 55 FEATURE(access_db, `hash -o -T<TMPF> /etc/mail/access')
 56 FEATURE(blacklist_recipients)
 57 FEATURE(local_lmtp)
 58 FEATURE(mailertable, `hash -o /etc/mail/mailertable')
 59 FEATURE(virtusertable, `hash -o /etc/mail/virtusertable')
 60
 61 dnl FEATURE(`genericstable')dnl
 62 FEATURE(`authinfo',`hash -o /etc/mail/authinfo/gmail.auth.db')dnl
 63
 64 dnl Google's SMTP servers need to do a DNS lookup on us (sending agent)
 65 MASQUERADE_AS(subdomain.mydomain.net)dnl
 66 FEATURE(masquerade_envelope)dnl
 67 FEATURE(masquerade_entire_domain)dnl
 68 MASQUERADE_DOMAIN(subdomain.mydomain.net)dnl
 69
 70 dnl Enable STARTTLS for receiving email.
 71 define(`CERT_DIR', `/etc/mail/certs')dnl
 72 define(`confSERVER_CERT', `CERT_DIR/mycert.pem')dnl
 73 define(`confSERVER_KEY', `CERT_DIR/mykey.pem')dnl
 74 define(`confCLIENT_CERT', `CERT_DIR/mycert.pem')dnl
 75 define(`confCLIENT_KEY', `CERT_DIR/mykey.pem')dnl
 76 define(`confCACERT', `CERT_DIR/cacert.pem')dnl
 77 define(`confCACERT_PATH', `CERT_DIR')dnl
 78 dnl WTF define(`confDH_PARAMETERS', `CERT_DIR/dh.param')dnl
 79
 80 dnl Uncomment to allow relaying based on your MX records.
 81 dnl NOTE: This can allow sites to use your server as a backup MX without
 82 dnl       your permission.
 83 dnl FEATURE(relay_based_on_MX)
 84
 85 dnl DNS based black hole lists
 86 dnl --------------------------------
 87 dnl DNS based black hole lists come and go on a regular basis
 88 dnl so this file will not serve as a database of the available servers.
 89 dnl For more information, visit
 90 dnl http://en.wikipedia.org/wiki/DNSBL
 91
 92 dnl Uncomment to activate your chosen DNS based blacklist
 93 dnl FEATURE(dnsbl, `dnsbl.example.com')
 94 dnl Alternatively, you can provide your own server and rejection message:
 95 dnl FEATURE(dnsbl, `dnsbl.example.com', ``"550 Mail from " $&{client_addr} " rejected"'')
 96
 97 dnl Dialup users should uncomment and define this appropriately
 98 define(`SMART_HOST', `[smtp.gmail.com]')dnl
 99 define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
100 define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl
101 define(`confAUTH_OPTIONS', `A p')dnl
102 TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
103 define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
104
105 dnl Uncomment the first line to change the location of the default
106 dnl /etc/mail/local-host-names and comment out the second line.
107 dnl define(`confCW_FILE', `-o /etc/mail/sendmail.cw')
108 define(`confCW_FILE', `-o /etc/mail/local-host-names')
109
110 dnl Enable for both IPv4 and IPv6 (optional)
111 DAEMON_OPTIONS(`Name=IPv4, Family=inet')
112 dnl DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')
113
114 define(`confMAX_HOP', 40)
115 define(`confBIND_OPTS', `WorkAroundBrokenAAAA')
116 define(`confNO_RCPT_ACTION', `add-to-undisclosed')
117 define(`confPRIVACY_FLAGS', `authwarnings,noexpn,novrfy')
118 MAILER(local)
119 MAILER(smtp)

I've honestly had a really rotten couple of days and I'm frustrated and I'm this close to just saying screw it and going back to letting the emails essentially fall into /dev/null, unless someone can spot something real simple that I've screwed up.
 
I'm assuming the at least problem is the From: line (it should say yourname@gmail.com,

Sure google will not accept other address than that, although it could. gpw928 told you how to solve this.

I think it is worth to try to solve your problem. I did it years ago, but I do not have the time to try now. But as said before, better you use other smpt acount, see for example this:

 
Dealing with Google's technical documents is always a hair-pulling exercise. Many of their technical documents feel like they are written by the marketing department.

It took me a good week to get the email running smoothly, and hruodr points out above that the method I describe (less secure apps) won't work past February 2021. And, apparently, Google think that the only two operating systems in the world are MacOS and Android...

Your mc file looks good. The domain name you masquerade as should be one that has a public DNS A and TXT records. The TXT record contains a key that Google provides to you. For gmail users, the domain name does not have to relate to anything currently used by your servers or your ISP. It's just got to exist, and resolve, and have a TXT record supplied by Google.
 
Perhaps you can use this to make a script to be run in /etc/aliases or .forward: mail/msmtp.

In principle not even that is necessary if your script makes the connection and do the smtp instructions.

I just discovered it, never used it. Perhaps it is simpler to configure that SMART_HOST in sendmail.

But I insist: better not to use google. Just use other account to mail to your google account.
 
Nice discussion as Im getting into mails now.

I have FreeBSD installed on my fileserver.

If there is so much trouble, maybe it would be better to acquire messages by FTP requests (to your pc f.e.). You could automate everything with some scripts and small bot so message would land at the end in your mailbox. You could also use other mail in between. I just think that if its already working file server, use its most trustworthy function.
Just a suggestion.
 
Back
Top