I am running an Apache2 web server (Server version: Apache/2.2.17 (Ubuntu)) on Ubuntu 11.04. I have an issue where long running downloads will fail after awhile. I have clients with slow connections which is how I first discovered the issue. If I download the file over my fast connection it will work fine. In testing I have been downloading a 3GB file while limiting my wget to 500K and get consistent failures. However, the amount of time or data transfered is always different.
I have two different Directory directives. Files will download fine from the one, but not the other. The main difference between the two is one is over NFS and has an XSendFile directive. Everything works fine with "download", but "vol1" has issues.
<Directory "download">
PassengerEnabled off
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<Directory "vol1">
PassengerEnabled off
Options -Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
XSendFile on
XSendFileAllowAbove on
</Directory>
Here is the failure from the client using wget:
HTTP request sent, awaiting response... 200 OK
Length: 3028287488 (2.8G) [video/mpeg]
33% [==============================> ] 1,025,449,932 295K/s in 48m 39s
2013-01-17 18:02:17 (343 KB/s) - Connection closed at byte 1025449932. Retrying.
After this wget will try again and sometimes get a 416 and sometimes a 200. In the access log example below I got a 200, but as you can see it thought the file was already downloaded because a content length of 250 was returned. I did notice sometimes on the initial download the wrong content-length was returned sometimes ~200MB sometimes in the kilobytes. As you can see the file I've been testing with is around 3GB.
Here is what the failure looks like in the access log:
[22/Jan/2013:13:59:22 -0500] "GET file.mpeg HTTP/1.1" 200 2533255750 "-" "Wget/1.13.4 (linux-gnu)"
[22/Jan/2013:15:22:02 -0500] "GET file.mpeg HTTP/1.1" 200 250 "-" "Wget/1.13.4 (linux-gnu)"
And for the times when it is a 416:
[22/Jan/2013:12:14:46 -0500] "GET file.mpeg HTTP/1.1" 200 1620176522 "-" "Wget/1.13.4 (linux-gnu)"
[22/Jan/2013:13:07:35 -0500] "GET file.mpeg HTTP/1.1" 416 638 "-" "Wget/1.13.4 (linux-gnu)"
I did find some online chatter about turning off SendFile when the directory is over NFS, but I have tested with other systems mounting a directory over NFS and they work just fine.
My NFS mount is mounted with the following directives:
defaults,vers=3,noauto
EnableSendFile offandEnableMmmap off. – David Schwartz Jan 22 at 21:04