In our application's configuration, nginx acts as a reverse proxy in front of
Our application replies to frontend requests with generally speaking, small responses... but some endpoints generate responses that are larger than one memory page (4K).
When this happens, nginx logs this warning:
an upstream response is buffered to a temporary file /path/to/nginx/proxy_temp/4/86/0000000864 while reading upstream, client: 22.214.171.124, server: api.ourdomain.com, request: "GET /pdf/..."
Our nginx logs end up flooded with this warning - and as far as I can see, the only solutions to make this warning vanish from our logs, are bad solutions:
I can set nginx
proxy_max_temp_file_sizeto 0 - basically disabling the buffering of large-ish responses. This would stop the buffering to files - but it would also mean that for endpoints generating large responses (e.g. PDF-generating ones that create 1-2MB responses) a slow consuming client would stall the corresponding gunicorn worker... In fact, if there are N gunicorn workers, it would only take N clients generating PDFs behind slow network connections, and our application would be down...
I can increase the
proxy_buffer_sizeto more than 4K (one memory page, that is). I am pretty sure this would have a severe impact in nginx performance - 70% of our responses indeed fit in 4K, and we would force nginx to allocate ... what? 2MB of buffers for each one of them, just to be ready for the occasional PDF-generating request? EDIT: In fact this is not an option at all - Michael (below) commented that the only allowed values are 4K and 8K.
I could turn
proxy_bufferingoff - but that is just as bad as the first solution (slow clients => death).
In essence, nginx floods our logs about something that we WANT - temporary buffering to files when the responses are big.
We just want to stop this from flooding our logs, by "marking" it as not really a warning (I am not even sure why this is a warning - what "bad thing" is it warning us about?)
Other than editing
nginx's source code and recompiling, is there any other solution that I am missing?