1

My setup: - nginx 1.0.6 ( compiled from scratch ) - php 5.3.8 with php-fpm ( also from scratch ).

I use this PHP script to stream files from the remote site to my users:

        $chunksize = 1 * (1024 * 1024); //you may want to change this
        $bytes_send = 0;
        if ($file = fopen($file, 'rb',false)) {      
            while(!feof($file) &&  (!connection_aborted()) && ($bytes_send<$new_length)) {

                $buffer = fread($file, $chunksize);            
                echo ($buffer);                       
                flush();
                $bytes_send += strlen($buffer); 
            }
            fclose($file);                         
        } else {
            die('Error - can not open file.');
        }

after i start to download with my connection ( 20 Mbps ) , i've got such result of ifstat

 eth0       
 KB/s in  KB/s out
 5105.77   1925.70
 5106.15   2063.77
 5083.80   2337.95
 4946.52   2487.28
 5127.37   2507.15
 5118.94   2474.98
 5172.55   2438.86
 4646.82   2451.87
 5246.18   2465.98
 5186.71   2459.66
 5032.03   2458.55

Is that normal? The result on the production server was a complete failure because of lack of bandwidth :| on apache the balance is even. Help really appreciated : /

@Edit Little benchmark made. Downloading 500 MB file via this. Results for about 250-260 MBs were just like that and then in gone down to almost 0. Is that some kind of network usage balance nginx does?

1 Answer 1

2

This happens because nginx tries to read from the backend (php-fpm in your case) as fast as possible, buffering the response first in memory (limited by the proxy_buffer_size and proxy_buffers settings), then on disk (up to the size specified in proxy_max_temp_file_size, default is 1 GiB). Therefore your ”streaming” script actually downloads the file as fast as it can, even if the client download speed is slower.

See the nginx http proxy module documentation — there are lots of parameters which could be tuned, and most of them can be set per location (so you can change the buffering behavior just for some URLs). You can even return the X-Accel-Buffering header from the backend to disable buffering, or, even better, return the X-Accel-Redirect header to make nginx stream the file itself without tying up a backend process.

3
  • About X-Accel-Redirect - Is it possible to be done, when backend is using some kind of context? ( Actually, it downloads from the remote site, but it sends some kind of headers - Cookies, POST values etc. ). Will it work by just redirecting?
    – Alex
    Sep 27, 2011 at 16:01
  • Fixed header values may be specified using proxy_set_header; POST request could be forced with proxy_method POST and parameters specified in proxy_set_body. If you need to send different values for different request, not sure that it would be possible. Sep 27, 2011 at 16:29
  • Well, setting the line: <?php header("X-Accel-Buffering: no"); ?> didn't made it. Any other ideas? I actually wonder how it works - I mean, in PHP it's fread(1*1024*1024); but what exactly determines, how much data my server will download from the remote site, and how to adjust it?
    – Alex
    Sep 27, 2011 at 16:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .