2

(I know 2013 is shortly out of support, and it will be replaced, but in the interim, I'm experiencing an issue I would like to fix.)

I will use telnet over port 25 as my example, but normal traffic also behaves the same way.

When I send a packet over port 25 to specific nodes in my Exchange cluster, it takes between 11 and 14 seconds before the banner is displayed with a 220 response. Since this affects emails too, it makes certain services rely on this communication time out.

Things I know:

  • About 70% of the nodes experience this issue fairly consistently. And the other 30% seemingly never exhibit this issue. They were all set up around the same time, but someone before my time might have changed settings on some nodes and not others. However, they "look" identical to me.
  • Tracing packets I can see the SMTP traffic arrive immediately on the target server. (No issues with network\latency, firewall, dns etc. everything flows smoothly)
  • I see an immediate TCP Accept on MSExchangeFrontendTransport.exe
  • After this packet has entered Windows and I've seen the TCP Accept, it takes the full 11-14 seconds before it shows up in any Exchange logs (SMTP Protocol Connector logs)
  • It's unknown when the issue was introduced, but it was likely a very long time ago, there just wasn't anything impacted by the delay before.

So, it seems like something is holding the traffic captive, but having turned down\off everything I can find and dug down into every imaginable log and process monitor results etc. I can't see anything impacting it.

Could this be internal to Exchange, or internal to a native Windows system process? And if there are no direct suggested solutions, where would you go from here to investigate this? I'm not entirely sure what would be the best way to proceed in the immediate future.

Any help will be greatly appreciated!

3
  • 1
    I'd suggest patching all the available updates for Exchange 2013 and see if there's any improvement.
    – Yuki Sun
    Commented Feb 7, 2023 at 8:38
  • Hi Yuki. They are already fully patched every month. No known missing patches.
    – wnogt
    Commented Feb 8, 2023 at 9:40
  • Hi wnogt. Then it's not an issue of missing patches...
    – Yuki Sun
    Commented Feb 13, 2023 at 7:32

2 Answers 2

4

The behavior you are witnessing might be a security feature called SMTP Tarpit. It slows SMTP responses on purpose to mitigate some attack types, like directory harvest attacks.

The setting is controlled by registry key HKLM\System\CurrentControlSet\Services\SmtpSvc\Parameters\TarpitTime The value is in seconds (DWORD)

Note: If the TarpitTime registry entry does not exist, Exchange behaves as if the value of this registry entry were set to 0. When the registry entry has a value of 0, there is no delay when the SMTP address verification responses are sent

More information: https://techcommunity.microsoft.com/t5/exchange-team-blog/smtp-session-tarpitting-for-windows-2003-and-exchange/ba-p/609829

2
  • 2
    Thanks for the suggestion! It seems like this key does not exist on the servers that experience the issue, or the ones that do not have issues. As an experiment I manually created the key to see if it would change anything. But that didn't seem to make a difference either. (I might be missing some context here, sorry if that's the case)
    – wnogt
    Commented Feb 7, 2023 at 9:15
  • Just make sure you're not using a 32-bit version of Exchange, and if you are, check for registry entries in the 32-bit registry view (under Wow6432Node)
    – canton7
    Commented Feb 7, 2023 at 19:59
4

A delay measured in seconds is often a timeout with a request to a third party. So something like a reverse DNS lookup of the incoming IP, or checking against a dead RBL also.

You can easily miss this if you are capturing only the packets between client and server. If it's not too huge an amount of data, try capturing everything* from the server around the time of the incoming connection and see if it's sending out requests that get no response. DNS would be my first guess but it's not the only possibility.

* Excluding things that common sense, the law, and your company's privacy policy disallow.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .