starttls fails after IP address change

Question

I have changed the IP address of our exim mail server (the virtual machine has been copied to a different host, also the network configuration is using NAT). Now it no longer accepts STARTTLS connection from a Windows Thunderbird client. It works with all other mail clients though.

The Thunderbird error message is something like can't set up a secure connection to mail.mydomain.de using STARTTLS, because the server is not offering this function. Disable STARTTLS or contact your provider. However the exim mail server does offer STARTTLS and it still works on the old system.

Here is an excerpt from the exim debug log:

27666 accept: condition test succeeded
27666 SMTP>> 220 s0107.mydomain.de ESMTP (Exim 4.77) Fri, 31 Oct 2014 05:23:27 +0100
27666 Process 27666 is ready for new message
27666 smtp_setup_msg entered
27666 SMTP<< EHLO [192.168.1.115]
27666 sender_fullhost = ip-123-123-123-123.hsi07.unitymediagroup.de ([192.168.1.115]) [123.123.123.123]
27666 sender_rcvhost = ip-123-123-123-123.hsi07.unitymediagroup.de ([123.123.123.123] helo=[192.168.1.115])
27666 set_process_info: 27666 handling incoming connection from ip-123-123-123-123.hsi07.unitymediagroup.de ([192.168.1.115]) [123.123.123.123]
27666 host in pipelining_advertise_hosts? yes (matched "*")
27666 host in auth_advertise_hosts? yes (matched "*")
27666 host in tls_advertise_hosts? yes (matched "*")
27666 SMTP>> 250-s0107.mydomain.de Hello ip-123-123-123-123.hsi07.unitymediagroup.de [123.123.123.123]
27666 250-SIZE 52428800
27666 250-PIPELINING
27666 250-AUTH PLAIN LOGIN CRAM-MD5
27666 250-STARTTLS
27666 250 HELP
27666 SMTP<< QUIT
27666 SMTP>> 221 s0107.mydomain.de closing connection
27666 LOG: smtp_connection MAIN
27666   SMTP connection from ip-178-200-216-217.hsi07.unitymediagroup.de ([192.168.1.115]) [123.123.123.123] closed by QUIT
27666 search_tidyup called
27654 child 27666 ended: status=0x0
27654 0 SMTP accept processes now running
27654 Listening...

On the original system the last few lines look like this:

2362 250-SIZE 52428800
2362 250-PIPELINING
2362 250-AUTH PLAIN LOGIN CRAM-MD5
2362 250-STARTTLS
2362 250 HELP
2362 SMTP<< STARTTLS
2362 setting SSL CTX options: 0x800
2362 tls_certificate file /etc/pki/tls/exim_tls/exim.cert
2362 tls_privatekey file /etc/pki/tls/exim_tls/exim.key
...

It seems to be a client issue, but I have no idea why the IP address change should have any effect on the STARTTLS handshake. Please advise.

Answer 1

You have confirmed above that more has changed than merely the IP address: specifically, the new network onto which the VM has been moved is behind a CISCO ASA firewall.

Those firewalls do what they describe as "protocol fixup", which in the case of SMTP means messing with the information that's passed at layer 4. By telnetting from your client to the server, and issuing an SMTP EHLO command, you have confirmed that although the server is advertising STARTTLS capability, the CISCO is removing this advertisement from the packets in flight. The client, not seeing STARTTLS being offered, and having been configured to require it, bails out.

You don't say whether the CISCO is under your control, but you have established that connections to port 587 on the server are not similarly affected (though I suspect that's an issue with CISCO configuration, and some future admin could easily turn it on). So you have, for the time being at least, a workaround.

Why CISCO think it's a good idea to strip encryption capabilities away from mail servers it protects is a mystery to me. But the upshot is that CISCOs in that mode are unfit-for-purpose when it comes to protecting mail servers.