0

(Note: I am not a networking engineer) We are sending files to an outside vendor and getting random timeouts on different services. It seems that we are getting timeouts most often on larger files. We did a packet capture that shows our window shrinking and suspect that the small payloads make it before the window hits 0, where large payloads give us a RST.

    11369 > su-mit-tg [ACK] Seq=677231 Ack=253694 Win=32768 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=256614 Win=29848 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=259534 Win=26928 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=262454 Win=24008 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=265374 Win=21088 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=268294 Win=18168 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=271214 Win=15248 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=274134 Win=12328 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=277054 Win=9408 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=279974 Win=6488 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=282894 Win=3568 Len=0
    11369 > su-mit-tg [ACK] Seq=677231 Ack=285814 Win=648 Len=0

Edit: I'm referring to different web-services that we are calling from our application. The timeouts don't consistently fail on a specific service, but instead hit all the services at different times. I cannot send it from a different network.

1
  • 1.What does this mean: getting random timeouts on different services? What do you mean by different services? Do you mean different file transfer methods? 2. Have you tried sending the files from a different network? If so, do you get the same results?
    – joeqwerty
    Mar 16, 2015 at 15:04

1 Answer 1

2

I think this problem related to IO problem related or application problem and for any reason the socket buffer has finished the space

I did something like this to reproduce the IO related problem in linux:

/dev/vdb                 2.0G  1.6G  470M  77% /brick1

[root@nod01 ~]# ls -l /dev/vdb
brw-rw---- 1 root disk 252, 16 Apr 19 22:46 /dev/vdb

echo "252:16 $((1024*250))" > /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device ## Limit write to 250KB per second

cd /brick1 ## change directory for downloading the Centos Iso


wget ftp://mirror.fdcservers.net/centos/6.4/isos/x86_64/CentOS-6.4-x86_64-bin-DVD2.iso


00:19:58.992042 IP mirror.50966 > nod01.example.com.46637: Flags [.], ack 1, win 46, options [nop,nop,TS val 2662018758 ecr 5131800], length 0
00:19:58.992107 IP nod01.example.com.46637 > mirror.50966: Flags [.], ack 11256736, win 0,  options [nop,nop,TS val 5144749 ecr 2661992655], length 0 ## I'm telling to the sender, please don't send me more data, because my socket buffer is full


[root@nod01 ~]# netstat -tunap | grep wget
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    

tcp   5264896      0 192.168.122.244:46637   208.53.158.34:50966     ESTABLISHED 15574/wget          #### note the sender has 5M of data in the doesn't buffer, because it cannot write fast in /brick1 as data arrive
tcp        0      0 192.168.122.244:51331   208.53.158.34:21        ESTABLISHED 15574/wget          

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .