117

I get the following error in my log files every time I try to upload a large file.

a client request body is buffered to a temporary file /var/lib/nginx/body/0000000001

Although the file uploads successfully, I always get the above error.

I increased the client_body_buffer_size to 1000m which is what I expect the largest file uploaded to be. However, this is was just a guess and although I don't get that error anymore I am wondering if this is an appropriate value to set for the client_body_buffer_size?

I would appreciate it if anyone can shed some light on this directive and how it should be used.

0

3 Answers 3

103

This is a warning, not an error. That's why it was prefaced with [warn] in the log.

It means that the size of the uploaded file was larger than the in-memory buffer reserved for uploads.

The directive client_body_buffer_size controls the size of that buffer.

If you can afford to have 1GB of RAM always reserved for the occasional file upload, then that's fine. It's a performance optimization to buffer the upload in RAM rather than in a temporary file on disk, though with such large uploads a couple of extra seconds probably doesn't matter much. If most of your uploads are small, then it's probably a waste.

In the end, only you can really make the decision as to what the appropriate size is.

8
  • 7
    Your answer has helped me make a decision. I will lower the value to about 512k to 1m. It's a shame I will get a lot of these warnings.
    – Abs
    May 29, 2013 at 20:53
  • 1
    Because of virtual memory, using a large value won't lead to "1GB of RAM always reserved for the occasional file upload". (No more RAM than it's actually needed for the current upload will be used.) Jun 2, 2018 at 7:38
  • 6
    If I set this to 50MB and had 200 people view a page at the same instant, would that take up 10GB of memory, or would the 50MB only be allocated for any users performing a file upload?
    – Codemonkey
    Sep 18, 2018 at 23:02
  • 2
    @Codemonkey This buffer is only used while a request body is being uploaded. Once the upload is complete the memory is free to be used for another request. And as the other commenter pointed out, while an upload is not in progress, no memory is used. So it depends on how many simultaneous uploads you have going on at any given instant. Sep 18, 2018 at 23:51
  • Often ten, probably never more than 20. 128GB box so I have plenty of memory..! Honestly though, I'd be doing it just to get rid of those [warn] lines in my error log - I should probably just ignore them!
    – Codemonkey
    Sep 19, 2018 at 1:14
39

If you don't want to NginX store the body content in a temporary file, you can set your config. like this:

    client_body_buffer_size     10M;
    client_max_body_size        10M;

If you set both of this configurations at the same max. size (in k, M or G for kB, MB or GB, respectively), you will prevent that NginX creates a temp. file.

For more info: http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size and http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size

7
  • 12
    But with that config you will also prevent all uploads which are larger than 10 MiB
    – Josef
    Feb 16, 2017 at 14:55
  • @Josef: Not prevent by any means. It is just that the request will need to be buffered to disk, and you will get the warning. Check the accepted answer.
    – OmarOthman
    Mar 7, 2017 at 15:18
  • 12
    @OmarOthman, Josef is right because of the client_max_body_size parameter. See the info on this link: client_max_body_size doc.: Sets the maximum allowed size of the client request body, specified in the “Content-Length” request header field. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.
    – eddy85br
    Mar 7, 2017 at 19:07
  • @SudipBhattarai It depends on what you are calling a temporary file, is it the NginX cache? Or this temp directory: nginx.org/en/docs/http/… ?
    – eddy85br
    Feb 27, 2020 at 15:27
  • 5
    @eddy85br Thanks. I had to do proxy_buffering off; to turn off response buffering and proxy_request_buffering off; to turn off request buffering. The proxy_cache off; is about caching response and it's turned off by default. :nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache Feb 27, 2020 at 17:57
12

If your upstream server that nginx reverse-proxies to supports streaming uploads of large files, it makes most sense to simply disable nginx buffering these uploads at all, using:

proxy_request_buffering off;
proxy_http_version 1.1;
client_max_body_size 0;

In case you also like to copy comments into your config:

# Explanation:
#     proxy_request_buffering off; -- upstream server can handle large streaming uploads
#     proxy_http_version 1.1;      -- required to disable request buffering also for clients that send chunked encoding
#     client_max_body_size 0;      -- otherwise nginx imposes a size restriction; only upstream should decide that
Directive details
  • Also use proxy_http_version 1.1 because as described in the proxy_request_buffering docs, request buffering will still be on in some cases:

    When HTTP/1.1 chunked transfer encoding is used [by the client], the request body will be buffered regardless of the directive value

Rationale for disabling request buffering

There is no point in nginx storing the request in such case:

  • It delays your upstream application seeing it.
    • This also means it cannot return errors early, such as "permission denied"; it's bad UX for the user to learn this only after waiting for a 30 minute upload.
  • It unnecessarily consumes additional disk space.
    • That introduces unexpected unreliability: People may expect that for a "passthrough-only" proxy server, only network and maybe RAM are important; they do not expect that requests will fail if the disk is too small.
  • Disk IO is often slow.

Buffering the entire request body is designed for upstream applications that cannot handle slow streaming uploads (e.g. PHP scripts with a maximum runtime duration). If your upstream app does not have such a restriction, you do not need request buffering to work around it.

Further links

  • See also this comment for a more detailed explanation of how nginx does proxy buffering, in particular proxy_request_buffering.

    In a link I put in there, an nginx developer explains why for (client <- nginx) connections, some caching can still be beneficial. But for (nginx <- upstream), "some caching" is not currently possible, as proxy_request_buffering always fully buffers the entire client request and does no streaming at all.

  • This post describes proxy buffering in the other direction (client <- nginx): https://www.getpagespeed.com/server-setup/nginx/tuning-proxy_buffer_size-in-nginx Check out especially the section Disable proxy buffering?, which has a similar argument as the one I have for (nginx -> proxy) above.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .