Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I changed the tcp initial window in my machine to 10 as shown below

[user@site etc]$ sudo ip route change default via 17.255.209.1 dev eth0  proto static initcwnd 10 

And changed tcp_slow_start_after_idle as shown below

[user@site etc]$ sudo sysctl -a | grep tcp_slow_start_after_idle
net.ipv4.tcp_slow_start_after_idle = 0

a ip route show confirmation is given below

[user@site etc]$ ip route show
default via 17.255.209.1 dev eth0  proto static  initcwnd 10
169.254.0.0/16 dev eth0  scope link  metric 1002
17.255.209.0/24 dev eth0  proto kernel  scope link  src 17.255.209.19

Now when I do a tcpdump on the website I don’t seem to see a change in the initial window with the WIN/MSS remaining 4 as default. 5840/1460=4

[user@site etc]$ sudo tcpdump -n -i any 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn and port 80'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
11:17:45.048174 IP 21.101.151.198.45873 > 17.255.209.19.http: Flags [S], seq 2008673341, win 5840, options [mss 1460,sackOK,TS val 1724223146 ecr 0,nop,wscale 6], length 0

The curl hit I did to the webpage requested around 30 KB of data.

[user@machine ~]$ curl http://www.site.com/js/main.js > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 88212  100 88212    0     0   179k      0 --:--:-- --:--:-- --:--:--  272k

What could be wrong in my approach?

Kernel

[user~]$ uname -r
3.0.4x86_64-linode21

As an update, here' are the results when I try google.com

[user@site ~]$ sudo tcpdump -n -i any 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn and host www.google.com'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
17:20:28.033236 IP 17.255.209.19.42799 > 74.125.127.106.http: Flags [S], seq 3148947324, win 14600, options [mss 1460,sackOK,TS val 193695310 ecr 0,nop,wscale 4], length 0

As you can see WIN/MSS is 14600/1460=10 in this case

I tried hitting my site from the server machine itself through curl and here's the result:

[user@site ~]$ sudo tcpdump -n -i any 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn and host www.site.com'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
17:25:14.584338 IP 17.255.209.19.35008 > 17.255.209.19.http: Flags [S], seq 3894567470, win 32792, options [mss 16396,sackOK,TS val 193981861 ecr 0,nop,wscale 4], length 0

WIN/MSS is 32792/16396=2 in this case

share|improve this question
    
keep in mind, if you are hitting this from a linux machine you will need to be on 3.0 as well, will dig through the source / tags to confirm the exact 3 version that instated the change –  Sam Saffron Mar 5 '12 at 12:43
    
@QuintinPar Could you add tcpdump output with connection outgoing from your test machine? –  kupson Mar 5 '12 at 16:59
    
@kupson I've updated the question –  Quintin Par Mar 5 '12 at 17:31
    
As far as I know you cannot influence Initial Window on incoming connections. Your connection to google shows that you have IW set to 10. Loopback inferface is quite special with that large MTU, maybe there is some cap on Initial Window in kernel source. –  kupson Mar 5 '12 at 17:40
    
keep in mind, 2 things will determine the IW. Clients maximal initial congestion window and Servers IW. The smaller wins. Run the test from 2 machines, the server should have the IW set by default in 3.0 ... and XP/Vista/Win7 clients do not restrict the IW so make good clients for the test. 3.0 Linux clients also work, but must be a separate machine. –  Sam Saffron Mar 6 '12 at 14:42
add comment

2 Answers

up vote 7 down vote accepted

I think you are misunderstanding how TCP works.

Each packet sent will always advertise a receiver window (aka. RWIN) and an optional scaling factor, see RFC 1323

The sender is not allowed to send more than the amount of data specified in the RWIN without it getting acknowledged. Depending on the congestion window, the sender may decide to fill up the RWIN or not.

So, there are two bits of information that are public in the TCP packets. The RWIN on the Server and the RWIN on the client. Both of these figures dictate what the maximal size of the congestion window can be on both ends.

The RWIN on the server is interesting when we are trying to optimise performance for say file uploads.

The RWIN on the client is interesting when we are trying to determine download speed.

Neither of these numbers make the congestion window on the other end public.

SO if I have an RWIN of 64k, the congestion window on the server can be ANY number lower than 64k.

The only way to determine what the actual congestion window is to count packets.

If I know:

  1. My round trip time (RTT) is ~200ms.
  2. I just requested a resource that is 100k.
  3. I have an RWIN of 64k.

If I get 2 packets back from the server that are 1452 bytes long within ~200ms, it is likely the congestion window on the server is smaller than 4356, cause if it were bigger 3 packets would be sent. If IW was set to 10, I would see a burst of 10 packets around the 200ms mark.

If you change your IW and want to confirm the change worked, you need to count packets to get an estimate on the congestion window size on the server.

Keep in mind, you probably want to look at the conversation directly after the SYN, SYN-ACK, ACK to ensure you are not looking at the middle of a conversation (where the congestion window could have already grown).

share|improve this answer
1  
The difference between the congestion window and the tcp window is stated in 20.6 (slow start) of TCP/IP Illustrated: "Slow start adds another window to the sender's TCP: the congestion window, called cwnd" (bold is mine). There is a sequence diagram in 20.7 that shows this at play during bulk transfer. –  Kyle Brandt Mar 12 '12 at 12:25
add comment

The window size will be whichever is smaller: server init window size or client RWIN. Since 5840 is the default RWIN for Linux 2.6, it seems that your client is the limiting factor here.

Try from a windows box. Windows XP has an RWIN of 64k, newer version 8k.

Source: http://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/ (The interesting part is below the video)

Edit: Expanding the answer to make it clearer:

  • In the TCP handshake, the client sends a SYN packet to the server sending its maximum allowed window size. (As your tcpdump output shows, these are 5840 bytes)
  • The server now responds with SYN ACK and the window size it would like to agree on. This window size can only be smaller than what the client proposed, not bigger. No matter how the server is configured, it can never have bigger window sizes than 5840 bytes with that client.
  • Client returns ACK and they happily exchange data ever after.

Edit2: The tcpdumps added to the question show the server opening connections to google and itself ACTING AS THE CLIENT.

share|improve this answer
    
I assume that's what Sam Saffron is referring to in his comment: Linux 3.0 has an RWIN of 10 MSS (14.600 bytes). –  Mantriur Mar 5 '12 at 16:01
    
Initial window (proposed in SYN packet) is 5840. It's a first packet and sending machine know nothing about receiver at this time (I tested it with "ip route flush cache"). –  kupson Mar 5 '12 at 16:18
    
Uh, 17.255.209.0 is your server subnet, right? The packet you are seeing is FROM 21.101.151.198.45873 TO 17.255.209.19.http. I am no expert on tcpdump output, but to me this reads: Hello Server, I am your client, I like 5840 byte windows. :) The next packet would be the server responding with ACK, 5840 is great, cheers. :) –  Mantriur Mar 5 '12 at 16:34
    
Just to emphasize, I think you got this the wrong way around: The first sending machine is the client as it opens the connection, not your server. It is the client offering the 5840 byte window. The server cannot propose a larger window size, only a smaller one. –  Mantriur Mar 5 '12 at 16:41
1  
I'm not original author of this question. I tested it (with similar results) on my own test environment and cannot change it either. Initial Congestion Window Size (initcwnd) has nothing to do with other side of connection. –  kupson Mar 5 '12 at 16:47
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.