Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am running an Apache2 web server (Server version: Apache/2.2.17 (Ubuntu)) on Ubuntu 11.04. I have an issue where long running downloads will fail after awhile. I have clients with slow connections which is how I first discovered the issue. If I download the file over my fast connection it will work fine. In testing I have been downloading a 3GB file while limiting my wget to 500K and get consistent failures. However, the amount of time or data transfered is always different.

I have two different Directory directives. Files will download fine from the one, but not the other. The main difference between the two is one is over NFS and has an XSendFile directive. Everything works fine with "download", but "vol1" has issues.

  <Directory "download">
      PassengerEnabled off
      Options Indexes MultiViews
      AllowOverride None
      Order allow,deny
      Allow from all
  </Directory>

  <Directory "vol1">
      PassengerEnabled off
      Options -Indexes MultiViews
      AllowOverride None
      Order allow,deny
      Allow from all
      XSendFile on
      XSendFileAllowAbove on
  </Directory>

Here is the failure from the client using wget:

HTTP request sent, awaiting response... 200 OK
Length: 3028287488 (2.8G) [video/mpeg]


33% [==============================>                                                               ] 1,025,449,932  295K/s   in 48m 39s

2013-01-17 18:02:17 (343 KB/s) - Connection closed at byte 1025449932. Retrying.

After this wget will try again and sometimes get a 416 and sometimes a 200. In the access log example below I got a 200, but as you can see it thought the file was already downloaded because a content length of 250 was returned. I did notice sometimes on the initial download the wrong content-length was returned sometimes ~200MB sometimes in the kilobytes. As you can see the file I've been testing with is around 3GB.

Here is what the failure looks like in the access log:

[22/Jan/2013:13:59:22 -0500] "GET file.mpeg HTTP/1.1" 200 2533255750 "-" "Wget/1.13.4 (linux-gnu)"
[22/Jan/2013:15:22:02 -0500] "GET file.mpeg HTTP/1.1" 200 250 "-" "Wget/1.13.4 (linux-gnu)"

And for the times when it is a 416:

[22/Jan/2013:12:14:46 -0500] "GET file.mpeg HTTP/1.1" 200 1620176522 "-" "Wget/1.13.4 (linux-gnu)"
[22/Jan/2013:13:07:35 -0500] "GET file.mpeg HTTP/1.1" 416 638 "-" "Wget/1.13.4 (linux-gnu)"

I did find some online chatter about turning off SendFile when the directory is over NFS, but I have tested with other systems mounting a directory over NFS and they work just fine.

My NFS mount is mounted with the following directives:

defaults,vers=3,noauto
share|improve this question
1  
This is specifically documented not to work for fundamental reasons. NFS and sendfile don't work together on Apache2. "I have tested with other systems mounting a directory over NFS and they work just fine." That's not how it works. If you cross the street without looking both ways and get hit by a car, you don't do that again. You don't try it on another street, get lucky and not get hit by a car, and then conclude that looking both ways wouldn't have helped! Use EnableSendFile off and EnableMmmap off. –  David Schwartz Jan 22 '13 at 21:04
    
Yes, I know about those settings and neither of them had any change on the outcome. I still get the errors. The reason I mentioned the other setup is because is almost exactly the same "street". –  bmckim Jan 23 '13 at 3:40
    
Any chance the file was truncated or modified while the request was being handled? –  David Schwartz Jan 23 '13 at 6:39
    
So that was a working theory I had and the file has not been modified since the 17th. –  bmckim Jan 23 '13 at 16:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.