5

I'm serving my website using Nginx as web server. I offer upload functionality to my users (they are allowed to submit pictures up to 5Mb) so I have the directive: client_max_body_size 5M; in my server config. What I've noticed is that if I try to upload any file the webserver does not prevent the upload of bigger files. Just as example, suppose I try to upload a really big video (a movie) of 700Mb. The server does not reject the upload instantaneously but it buffers the whole data (taking so long and slowing down the server) and only at the end of the upload it returns a 413 Request entity too large error.

So the question is: is there any way to properly configure Nginx to block big files upload when the transmitted data starts to overcome my client_max_body_size limit?

I think it would be really insecure to go on production with my actual settings and I wasn't able to find anything helpful on google.

Edit:

I'm using php and Symfony2 as backend...

Edit (again):

This is what appears in my error.log:

2013/01/28 11:14:11 [error] 11328#0: *37 client intended to send too large body: 725207449 bytes, client: 33.33.33.1, server: www.local.example.com, request: "POST /app_dev.php/api/image/add/byuploader HTTP/1.1", host: "local.example.com", referrer: "http://local.example.com/app_dev.php/"`

The weird things is that i am monitoring my nginx error.log with tail -f error.log and the message appears immediately when the upload starts (before it ends). So nginx makes some kind of preventive check, but it doesn't stop/chunk the upload request...

I also tried to verify if php gets the control of the upload by issuing a echo 'something'; die(); on the page who handles the upload but it didn't printed out anything and didn't stopped the upload request. So it should be nginx or some php internals that make the whole upload continue until it's completely transmitted...

another edit: here's a view of my top (monitoring nginx and php) when a big file upload is in progress (nginx gets overloaded): top: monitoring php and nginx

edit: here's my nginx configuration

server {
    listen   80;
    server_name www.local.example.com local.example.com;
    access_log /vagrant/logs/example.com/access.log;
    error_log /vagrant/logs/example.com/error.log;
    root   /vagrant/example.com/web;
    index app.php;
    client_max_body_size 6M;

    location /phpmyadmin {
        root /usr/share;
        index index.php;
        location ~* \.php {
            fastcgi_intercept_errors on;
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_split_path_info ^(.+\.php)(/.*)$;
            include fastcgi_params;
            fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
            fastcgi_param  HTTPS              off;
        }
    }

    location / {
        try_files $uri @rewriteapp;
    }

    location @rewriteapp {
        rewrite ^(.*)$ /app.php/$1 last;
    }

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    location ~ ^/(app|app_dev)\.php(/|$) {
        fastcgi_intercept_errors on;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
        fastcgi_param  HTTPS              off;
    }
}
16
  • 4
    Are you sure it's Nginx that's buffering the file? Jan 28, 2013 at 10:46
  • No, how can I be sure of this? Jan 28, 2013 at 10:50
  • Updated my question with a specific test to verify if it's an nginx or php behavior that handles upload this way... Jan 28, 2013 at 11:23
  • 2 things, if you want to make a image uploader, you sould check the fileextension and only allow image extencions. you should doublecheck the size. it sounds like php does not give up and forces somehow to load up the file. Jan 28, 2013 at 11:27
  • 1
    @LucianoMammino: Is is the nginx process that's using the memory? It's memory consumption that's the problem, right? Jan 28, 2013 at 11:37

2 Answers 2

3

Useful information here: https://stackoverflow.com/questions/4947107/nginx-upload-client-max-body-size-issue

I quote part of the most interesting answer (for my case):

Most clients don't read for responses until the entire request body is sent. Because nginx closes the connection the client sends data to the closed socket, causing a TCP RST.

If so it's only a browser problem that will not affect (overload) my web server. It seems to complain with my memory inspection that showed both nginx and php-fastcgi haven't been truly overloaded when uploading a large (700mb) file.

I could address the browser problem in several ways, for example:

  • add a <input type="hidden" name="MAX_FILE_SIZE" value="5242880" /> in my form
  • use a custom 413 nginx error page
0

What are you using php or rails or something? because you should check the size before uploading.
set post_max_size and upload_max_filesize to corresponding value in file:php.ini

edit: try to write something like this to your upload

$element = new Zend_Form_Element_File('foo');
$element->setLabel('Upload a file:')
        ->setDestination('/var/www/upload');
// make sure its only 1 file
$element->addValidator('Count', false, 1);
// Maximal 100k
$element->addValidator('Size', false, 102400);
// only JPEG, PNG, and GIFs
$element->addValidator('Extension', false, 'jpg,png,gif');
$form->addElement($element, 'foo');
5
  • I have the following settings for these 2 config values: upload_max_filesize => 5M => 5M and post_max_size => 8M => 8M (taken with php -i from command line) Jan 28, 2013 at 10:54
  • have a look at the .htaccess in the root folder of you website. there might be some more modifications to. the definitions in the .htaccess are more dominant IF there are some. Are you hiding your Website behind a proxy? Jan 28, 2013 at 10:56
  • I do use nginx so no .htaccess files :) Jan 28, 2013 at 10:58
  • 1
    I would reupload a large sized file and have a look at the log's. try to find the message where it stops to upload and have at this information. Feel free to post it here so we might find out a little more Jan 28, 2013 at 11:05
  • Just updated my question to add additional informations about what appears in my logs Jan 28, 2013 at 11:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .