10

I've searched everywhere and I'm really struggling with this one. I think I've tried just about everything.

Background info

  • VPS with CentOS 6.7
  • Postfix 2.6.6
  • dovecot, amavis, mysql, fail2ban
  • I've verified with my VPS provider that they do not block any ports.

Things I've done

  • Removed stock sendmail
  • I've installed postfix, dovecot, mysql etc. for a complete mail solution
  • I am allowing only imap, smtp with STARTTLS (ports 143 and 587)
  • SSH logins disabled, only with keys
  • I can receive mails (via port 143)
  • I can telnet from localhost to both ports (587, 25) and I get postfix greeting
  • Trying to connect to 587 or 25 (mail client or telnet) gets me zero response i.e. connection timeout

Things I've tried

1) Are ports open? Yes, iptables:

Chain INPUT (policy DROP 11 packets, 1375 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:587
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:587
   25  2579 f2b-dovecot  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport dports 110,995,143,993,587,465,4190
   68  7788 f2b-postfix  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport dports 25,465,587
    0     0 f2b-sshd   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport dports 22
   25  2579 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:143
    7   600 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:8080
23464 2662K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
   49  2940 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           icmp type 8
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:3915
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:80
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:443
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:143
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:587
    5   300 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:25

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 23235 packets, 2494K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain f2b-dovecot (1 references)
 pkts bytes target     prot opt in     out     source               destination
   25  2579 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-postfix (1 references)
 pkts bytes target     prot opt in     out     source               destination
   68  7788 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain f2b-sshd (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

2) Is postfix listening on port 587? Yes. Is it listening on localhost only? No, any host.

Here's netstat:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      7173/master
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN      917/httpd
tcp        0      0 127.0.0.1:4190              0.0.0.0:*                   LISTEN      749/dovecot
tcp        0      0 0.0.0.0:587                 0.0.0.0:*                   LISTEN      7173/master
tcp        0      0 0.0.0.0:143                 0.0.0.0:*                   LISTEN      749/dovecot
tcp        0      0 127.0.0.1:24                0.0.0.0:*                   LISTEN      749/dovecot

And here's the postfix/main.cf as well, just in case:

# Enable both IPv4 and/or IPv6: ipv4, ipv6, all.
inet_protocols = ipv4

# Enable all network interfaces.
inet_interfaces = all

3) Are you forcing secure connections correctly? To the best of my knowledge, yes, here's the postfix/master.cf:

# Submission, port 587, force TLS connection.
submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o content_filter=smtp-amavis:[127.0.0.1]:10026

4) What about smtpd restrictions? Seems ok:

# HELO restriction
smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    reject_non_fqdn_helo_hostname
    reject_invalid_helo_hostname
    check_helo_access pcre:/etc/postfix/helo_access.pcre

5) Is postfix even working right?

Yes, logging into server and sending test mail from console works and e-mail is received on the other end, i.e.:

echo "Test mail from postfix" | mail -s "Test Postfix" [email protected]

6) What happens when connecting on port 587?

Apparently, nothing at all. If I try to telnet to any random port, I at least get something. For example, trying to telnet to port 666 (which isn't open) produces no reply to the client, but at least I get something in tcpdump:

15:22:20.305697 IP xxx > xxx.com.mdqs: Flags [S], seq 3195304468, win 8192, options [mss 1352,nop,wscale 8,nop,nop,sackOK], length 0

When tcpdumping port 587, absolutely nothing at all happens when trying to telnet to it.

What am I still missing?

Everything above exhausts my knowledge of the things I can still try. I've managed to nail it down to my port 587 being completely blocked off by something. As I've said, my VPS provider confirmed that they are not blocking any ports at all. I've tried port 25 and it's the same story.

The only other thing I can see is that I have somehow blocked those ports when setting up my server, but I can't recall if that's the case and I don't know how to test for that.

I'd really appreciate any help you can give me. In fact, I'm buying a beer for whoever helps me solve this, I've already wasted two days on it and it's starting to get really annoying.

5
  • 3
    Could you replace the iptables output above with that of iptables -L -n -v, entire and unredacted? Also, given your tcpdupmp testing, and your provider's assurance that i doesn't block 25 and 587 inbound, it's also possible that the place you're testing from blocks 25 and 587 outbound. If you posted the IP address it would be a lot easier to confirm or deny that hypothesis.
    – MadHatter
    May 19, 2016 at 13:46
  • Thanks for replying. Sure, I've edited my post. Hmm, I've never thought about ports being blocked on my end, that's a good idea. I'll try and test this. May 19, 2016 at 14:18
  • 1
    @MadHatter, you were completely right on blocked ports on my side. It turned out (after a lot of calls) that admin of my company's VPN decided to start blocking certain ports, 25 and 578 being one of them, but not others. This explains why I never had such problems before and didn't even think about it. My server was probably already working fine the last couple of days while I was losing my mind over it. If you want, you can post as a real answer and I'll accept it. Also, please PM me your PayPal (or similar) info, you really deserve a large cold one, god knows when I'd figure it out. May 19, 2016 at 18:40
  • I've written it up, so you can accept the answer - because otherwise the question will float around for ever, and that's bad. Also, this might well be of use to others in the future. I meant what I said in my answer about your having done most of the investigative work already (more power to you!) but if you feel honour-bound to send me one beer, you can find my contact details on my personal website, via my profile.
    – MadHatter
    May 19, 2016 at 19:41
  • Done and thank you again for your help, you likely saved me hours if not days of wasted time. I'll contact you tomorrow, already checked your website. May 19, 2016 at 20:23

1 Answer 1

4

Reachergilt, welcome to SF, and thanks for an excellent first question. You may feel that we've saved you, but honestly - you'd already done all the heavy lifting, and presented it very systematically. With a forensic mindset like that, I hope you stay around these parts for some time to come.

The tcpdump output is particularly damning. It proves beyond reasonable doubt that your attempts to connect aren't even reaching your server, which exonerates the server's firewall, postfix's bindings, and everything else server-side.

Armed with that pointer, you've gone away and confirmed that your outbound connections were being blocked, so your tests were never getting as far as your server. It is fairly (depressingly) normal for 25 (and to lesser extent, 587) to be blocked outbound from modern networks, because spam-sending botnets use them.

At any rate, you are now happy that your new mail server is working as advertised, and that's good.

1
  • 3
    Yeah, my post does make a nice checklist for anyone having similar problems. Funny enough, because I thought something was wrong with my server, I quadruple checked everything from A to Z, allowing me to document everything here. May 19, 2016 at 20:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .