0

I have a client who uses an external mail server to log and filter their inbound email before passing it on to their main mail system (which is Office365). The external server is running Sendmail, formerly on Debian, now on Ubuntu (Amazon's version).

The change of OS (and hosting) has also involved a bump in the version of Sendmail, from 8.14.4 on the older server (quite elderly, yes) to 8.15.2 on the newer one. Unfortunately it's also induced a slight change in behaviour and I am struggling to find whether this is controlled by a flag or other config setting.

The external server does some filtering of spam and other cruft and, up until the change of server, all of the remaining mail was successfully delivered to O365. (Some of it was marked as spam there, but it all got to their servers at least.) This is no longer true. Most of the mail which is being held, and not delivered, is bounce messages from their mailshots, but they do need to see those to clean their lists.

It seems that the mail which is not being delivered is mail which has names which are not fully qualified in the mail headers. They aren't from the hop where it arrives on the external server -- we would reject e.g. nonexistent domains at that stage -- but from earlier on in the mail's journey. So we might see headers like

    Received: from EXTERNAL.com (EXTERNAL.com [NN.NN.NN.NN])
        by OUR.SERVER (8.15.2/8.15.2/Debian-10) with ESMTP id xELIDED
        for <VALID@OUR.DOMAIN>; Sat, 11 May 2019 10:49:17 GMT
    Received: from EXTERNAL.com.local (EXTERNAL.com [NN.NN.NN.NN])
        by EXTERNAL.com (8.16.0.22/8.16.0.22) with ESMTPS id xELIDED
        (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT)
        for <VALID@OUR.DOMAIN>; Sat, 11 May 2019 18:49:15 +0800

-- it is the .local header which is causing the problems. But we aren't seeing these problems when the mail arrives at the server, only when it leaves the server to head to Office365. At that point, we see

    EXTERNAL.com.local: Name server timeout

And the transaction concludes with

    timeout writing message to OURDOMAIN-com.mail.protection.outlook.com.
<VALID@OUR.DOMAIN>... Deferred: Name server: OURDOMAIN-com.mail.protection.outlook.com.: host name lookup failure
Closing connection to OURDOMAIN-com.mail.protection.outlook.com.

i.e. the eventual failure message suggests that it's failing to connect to O365. This isn't the case since the connection starts successfully, other connections to the O365 servers are connecting successfully (this is a moderately high-traffic server).

Sendmail logging and tcpdump shows that the initial part of the SMTP connection goes fine. In the DATA section, the headers are transmitted and then the connection terminates due to not being able to look up a hostname which is not immediately relevant to the connection (it never gets as far as transmitting any of the body of the email).

As well as situations where there was a hostname within the headers which could not be resolved, I've also seen this happen on an email where the reply-to had been set to a domain which did not exist and was thus not resolvable. (Quite likely spam but that isn't really the issue here.)

I've looked through the change logs for the relevant versions of Sendmail and haven't spotted anything obviously relevant; I've also spent a little bit of time poking through the source code. The logs from tcpdump show the connection being terminated on our side, not Microsoft's -- we have an engineer from their side attempting to help us but, because there is never a successful connection for these mails, they're struggling to see what is going on. DNS lookups for everything else seem to be working fine.

If anyone knows where to find the config which will say "don't try to look up irrelevant hostnames", I'm all ears. These aren't our misconfigurations!

Thanks in advance.

Edit Adding in a bit of the strace log:

    connect(9, {sa_family=AF_UNIX, sun_path="/dev/log"}, 110) = 0
    sendto(9, "<22>May 15 09:52:20 sendmail[135"..., 176, MSG_NOSIGNAL, NULL, 0) = 176
    >>> EHLO our.server.name
    250-VE1EUR02FT009.mail.protection.outlook.com Hello [NN.NN.NN.NN]
    250-SIZE 157286400
    250-PIPELINING
    250-DSN
    250-ENHANCEDSTATUSCODES
    250-8BITMIME
    250-BINARYMIME
    250-CHUNKING
    250 SMTPUTF8
    >>> MAIL From:<> SIZE=23108
    250 2.1.0 Sender OK
    >>> RCPT To:<VALID@OUR.DOMAIN>
    >>> DATA
    250 2.1.5 Recipient OK
    354 Start mail input; end with <CRLF>.<CRLF>
    socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 12
    connect(12, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, 16) = 0
    sendto(12, "R>\1\0\0\1\0\0\0\0\0\1\2the\4dodgy\3com\5local\0\0"..., 46, MSG_NOSIGNAL, NULL, 0) = 46
    recvfrom(12, "R>\201\202\0\1\0\0\0\0\0\1\the\4dodgy\3com\5local\0\0"..., 8192, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, [28->16]) = 46
    [repeats six more times]
    the.dodgy.com.local: Name server timeout
    timeout writing message to OURDOMAIN-com.mail.protection.outlook.com.
    sendto(9, "<18>May 15 09:52:20 sendmail[135"..., 130, MSG_NOSIGNAL, NULL, 0) = 130
    <VALID@OUR.DOMAIN>... Deferred: Name server: OURDOMAIN-com.mail.protection.outlook.com.: host name lookup failure
    sendto(9, "<22>May 15 09:52:20 sendmail[135"..., 296, MSG_NOSIGNAL, NULL, 0) = 296
    Closing connection to OURDOMAIN-com.mail.protection.outlook.com.
    +++ exited with 70 +++

So the lookup that fails is the one on the.dodgy.com.local (referred to as EXTERNAL.com.local in the earlier sanitised log but I wanted to preserve the structure of the function calls here), but the one that is reported back by Sendmail is a failure of a lookup on the server which it was connected to at the time. It isn't doing that lookup and it wouldn't have failed if it did.

Edit (2): changing the DNS resolver does not help.

  • You have general DNS problems, not specific to any host. Check what DNS servers the instance has configured in /etc/resolv.conf. They are probably broken. – Michael Hampton May 15 at 17:09
  • Unfortunately this isn't the case. All lookups that should work -- that is, for names which actually exist in global DNS -- are working, and it's only lookups for names which do not exist which are failing. The point here is that these names shouldn't be being looked up in the first place, as they're not relevant to the processing of the email. In this case the DNS setup is the standard one that comes with (this type of) Amazon server; we've not touched it at all and have no reason to think it would be misconfigured. – meldh May 15 at 18:05
  • Are you saying that OURDOMAIN-com.mail.protection.outlook.com. doesn't actually exist then? – Michael Hampton May 16 at 1:25
  • No, but Sendmail is reporting the "wrong" error. strace on an instance of Sendmail with one of these affected messages shows that it gets as far as the DATA part of the transaction, then attempts to look up the nonexistent .local name (several times). At that point it logs the EXTERNAL.com.local: Name server timeout error mentioned above, and terminates complaining about a nameserver timeout on OURDOMAIN-com.mail.protection.outlook.com., but the timeout on the legit domain never actually happens. – meldh May 16 at 7:31
  • (Not only is there no additional lookup happening on the legit domain per strace, it's connected to that host at the time anyway -- or to one of those hosts, I should say, since it resolves to a pool of servers at the MS end.) – meldh May 16 at 7:32
1

I saw a link to Why is sendmail calling dns_getcanonname for domains of non-recipients in the To: header? in the sidebar and it turned out to have the solution. We needed to add FEATURE(nocanonify', canonify_hosts') in sendmail.mc and that appears to have stopped the additional lookups; the mail that was previously failing to be delivered has now gone through.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.