Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

A customer is trying to send emails with (smaller and larger) attachments to one of our exchange servers, but get the connection reset after timeout is met. To me, it seems that the sending server does not receive the ACKs, and hence resends, resulting i DUP ACK from our side. We are using Cisco ASA, but we're not using any smtp/esmtp policies (aka fixup smtp) for any of the involved interfaces (it is used for a completely different vlan, where no exchange resides).

1.1.1.1 receiving smtp server
2.2.2.2 sending smtp server

Up to and including the "S: 354 Start mail input; end with .", it works like a charm. The problem really comes when data is sent.

Wireshark dump

No   Time        Source                Destination           Protocol Length Info
20   504.923698  1.1.1.1         2.2.2.2         SMTP     78     S: 250 2.1.5 Recipient OK

No   Time        Source                Destination           Protocol Length Info
21   505.304394  2.2.2.2         1.1.1.1         SMTP     60     C: DATA

No   Time        Source                Destination           Protocol Length Info
22   505.304713  1.1.1.1         2.2.2.2         SMTP     100    S: 354 Start mail input; end with .

No   Time        Source                Destination           Protocol Length Info
23   505.599857  2.2.2.2         1.1.1.1         SMTP     1434   C: DATA fragment, 1380 bytes

No   Time        Source                Destination           Protocol Length Info
24   505.620808  2.2.2.2         1.1.1.1         SMTP     1434   C: DATA fragment, 1380 bytes

No   Time        Source                Destination           Protocol Length Info
25   505.620823  1.1.1.1         2.2.2.2         TCP      54     smtp > 55346 [ACK] Seq=450 Ack=2904 Win=64860 Len=0

No   Time        Source                Destination           Protocol Length Info
26   505.919899  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Previous segment lost] C: DATA Fragment, 1380 bytes

No   Time       Source                Destination           Protocol Length Info
27   505.919912  1.1.1.1         2.2.2.2         TCP      54     [TCP Dup ACK 25#1] smtp > 55346 [ACK] Seq=450 Ack=2904 Win=64860 Len=0

No   Time        Source                Destination           Protocol Length Info
28   505.940785  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Previous segment lost] C: DATA fragment, 1380 bytes

No.  Time        Source                Destination           Protocol Length Info
29   505.940797  1.1.1.1         2.2.2.2         TCP      54     [TCP Dup ACK 25#2] smtp > 55346 [ACK] Seq=450 Ack=2904 Win=64860 Len=0

No.  Time        Source                Destination           Protocol Length Info
30   505.961793  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Retransmission] C: DATA fragment, 1380 bytes

No.  Time        Source                Destination           Protocol Length Info
31   505.982494  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Retransmission] C: DATA fragment, 1380 bytes

No.  Time        Source                Destination           Protocol Length Info
32   505.982508  1.1.1.1         2.2.2.2         TCP      54     smtp > 55346 [ACK] Seq=450 Ack=4284 Win=64860 Len=0

No.  Time        Source                Destination           Protocol Length Info
33   506.302829  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Previous segment lost] C: DATA fragment, 1380 bytes

No.  Time        Source                Destination           Protocol Length Info
34   506.302846  1.1.1.1         2.2.2.2         TCP      54     [TCP Dup ACK 32#1] smtp > 55346 [ACK] Seq=450 Ack=4284 Win=64860 Len=0

No.  Time        Source                Destination           Protocol Length Info
35   506.323446  2.2.2.2         1.1.1.1         SMTP     1434   [TCP Retransmission] C: DATA fragment, 1380 bytes

etc etc until timeout met.

We run other exchange servers, to which the sender can send the very same email to. All of our exchange servers sit behind the same firewalls, routers and switches. Probably only the patch cabling that diffs.
oh, and sending attachments on 15MB from gmail to the server works

Normal continous ping:

Ping statistics for 2.2.2.2:
    Packets: Sent = 249, Received = 249, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 82ms, Maximum = 546ms, Average = 138ms
^C

# unfragged packet of 992 bytes works
 C:\Users\someadmin>ping -f -l 992 2.2.2.2

Pinging 2.2.2.2 with 992 bytes of data:
Reply from 2.2.2.2: bytes=992 time=100ms TTL=48
Reply from 2.2.2.2: bytes=992 time=101ms TTL=48
Reply from 2.2.2.2: bytes=992 time=101ms TTL=48
Reply from 2.2.2.2: bytes=992 time=100ms TTL=48

Ping statistics for 2.2.2.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 100ms, Maximum = 101ms, Average = 100ms


# unfragged packet of 993 bytes fail
C:\Users\someadmin>ping -f -l 993 2.2.2.2

Pinging 2.2.2.2 with 993 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 2.2.2.2:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

I can however ping googles dns with large packets:

ping -f -l 1472 8.8.8.8

Pinging 8.8.8.8 with 1472 bytes of data:
Reply from 8.8.8.8: bytes=64 (sent 1472) time=31ms TTL=51

ping -f -l 1472 8.8.4.4

Pinging 8.8.4.4 with 1472 bytes of data:
Reply from 8.8.4.4: bytes=64 (sent 1472) time=30ms TTL=51

Cisco ASA policies

class-map inspection_default
 match default-inspection-traffic
!
!             
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 3096
  no dns-guard
  no protocol-enforcement
  no nat-rewrite
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map 
  inspect ftp 
  inspect h323 h225 
  inspect h323 ras 
  inspect rsh 
  inspect rtsp 
  inspect sqlnet 
  inspect skinny  
  inspect sunrpc 
  inspect xdmcp 
  inspect sip  
  inspect netbios 
  inspect tftp 
  inspect ip-options 
  inspect icmp 
policy-map shape_policy
 class class-default
  police input 10000000 5000
  police output 10000000 5000
!             

Where should I start looking? Should I start by require the sender to do the same wireshark/tcpdump trace?

share|improve this question
    
where are those pings originating? From the same LAN that the server is on? –  joeqwerty Jun 27 '11 at 14:32
    
No, across internet. –  3molo Jun 27 '11 at 14:42
add comment

1 Answer

It's hard to know for sure but, in my opinion, you have a MTU path issue here. Do a path MTU discovery and reduce the MTU of your gateway (or server NIC) accordingly. If it solves your problem, then you have your proof that some node in the path isn't handling MTU correctly (either dropping the ICMP code 4 packets or simply not sending it back).

share|improve this answer
    
Updated where I proove I have no problem pinging google dns with 1472 byte packets. Does that rule out a problem on my side (and in fact the first few hops from me to him), in your opinion? –  3molo Jun 27 '11 at 14:34
    
Well, if it's between you and a specific nod, chances are the problem is on their side or, at least, closer to them than to you. Performing a path MTU discovery is easy with windows, although a bit time-consuming. Check out the -l and -f parameters of the ping command and practice your divide and conquer algorithmic –  Stephane Jun 27 '11 at 15:15
    
I already did. 993 bytes or more results in no response –  3molo Jun 28 '11 at 7:55
    
Ok then set the MTU on your server NIC to 992 and that should solve your issue. Note that you might want to contact the operator of the remote server and let him know that they have a problem with their path and that they should look into this: having such a short path MTU isn't too much of a problem as long as you handle ICMP code 4 properly at all steps and as long as your equipment handles fragmented packet properly. –  Stephane Jun 28 '11 at 8:07
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.