13

I use nginx 1.2.3 to proxy to a script:

proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8880;
proxy_buffering off;
proxy_read_timeout 300s;
gzip off;

The scripts sends both Transfer-encoding: chunked and Content-Length: 251:

HTTP/1.0 307 Temporary Redirect
Content-length: 251
Pragma: no-cache
Location: /...
Cache-control: no-cache
Transfer-encoding: chunked

I need both, but nginx automatically removes the Content-Length:

HTTP/1.1 302 Found
Server: nginx/1.2.3
Content-Type: application/json; charset=utf-8
Content-Length: 58
Connection: keep-alive
Location: /...

As a result, the clients do not wait for the chunks to be sent. This used to work with an earlier version of nginx.

4
  • What do the headers look like from the nginx proxy?
    – hrunting
    Feb 28, 2013 at 4:10
  • which version did it used to work with?
    – cnst
    Mar 5, 2013 at 9:35
  • It used to work with nginx 0.9.8
    – Julien
    Mar 6, 2013 at 3:53
  • 1
    You're violating HTTP protocol. It works with nginx 0.9.8, because till the 1.1.4 version it doesn't support chunked encoding at all.
    – VBart
    Sep 30, 2013 at 22:05

5 Answers 5

14

Unfortunately, I can't comment on cnst's post - so I'm going to answer here.

The nginx_http_proxy module by default talks with the upstream in HTTP/1.0. This can be changed with the directive proxy_http_version 1.1.

This might also be the cause for your script to return a HTTP/1.0 answer, although chunked coding and status code 307 don't exist in this version.

You shouldn't use chunked coding with a redirect either, as this doesn't really make sense.

Additionally, it seems like nginx doesn't pass chunks from the upstream to the client one by one, but it buffers the upstream's response. The Content-Length header field is ignored because it is against the definition. I had to look at the source code of the module because all this appears to be undocumented.

You may want to try out the nginx_tcp_proxy_module to proxy the chunked content as raw TCP data: Module at Github


UPDATE (10.04.14)
The nginx_http_proxy module has support for X-Accel-* headers, of which one (X-Accel-Buffering: yes|no) controls whether the response should be buffered or not.

Adding this header (X-Accel-Buffering: no) to the backend's response will cause nginx to directly pass chunks to the client.

This header allows to control buffering on an per-request basis.

The module also has a configuration directive proxy_buffering to enable or disable response buffering (not buffering means sending chunks will work).

Proxy buffering (both header and directive based) is documented here.

3
  • He should not do that even with nginx_tcp_proxy_module. It works with some browsers only because they are very error tolerant.
    – VBart
    Sep 30, 2013 at 22:08
  • because all this appears to be undocumented Wrong. It's documented in RFC 2616. See 13.5.1.
    – VBart
    Sep 30, 2013 at 22:10
  • @VBart Sure there are standards - but there's only very few information about how far particularly nginx implements them.The TCP proxy module is hust a suggested workaround.
    – Lukas
    Sep 30, 2013 at 23:27
11

As Lukas alludet to, HTTP 1.1 prohibits Content-Length if there's a Transfer-Encoding set.

Quoting http://www.ietf.org/rfc/rfc2616.txt:

   3.If a Content-Length header field (section 14.13) is present, its
     decimal value in OCTETs represents both the entity-length and the
     transfer-length. The Content-Length header field MUST NOT be sent
     if these two lengths are different (i.e., if a Transfer-Encoding
     header field is present). If a message is received with both a
     Transfer-Encoding header field and a Content-Length header field,
     the latter MUST be ignored.
1
  • 2
    Additionally, Nginx's correct behaviour in compliance with HTTP 1.1 goes a long way towards preventing HTTP request smuggling attacks. Mar 14, 2020 at 13:25
2

You have not specifically elaborated why your script needs chunked encoding in the first place, especially with a redirect response.

I see a multitude of problems here.

  • Transfer-Encoding: chunked is an HTTP/1.1 feature (and your script seems to be replying with an HTTP/1.0 header)

  • there is no 307 in HTTP/1.0

  • the whole purpose of chunked is that you don't know what your Content-Length would have been, so, chunked is used in place of providing the length within Content-Length, where instead lengths are provided within the body of the response, intermixed with the actual content; it would be pointless for a script to be generating both headers upfront

I'm not personally familiar with chunked, but as per the basic info at http://en.wikipedia.org/wiki/Chunked_transfer_encoding and also https://www.rfc-editor.org/rfc/rfc2616#section-3.6.1, I would guesstimate that your script's whole handling of the chunked encoding may be completely wrong.

If the above still doesn't cover it, and in all actuality otherwise, it is also unclear why a reply with a 307 or 302 http status code should be provided with a "weird" encoding. There was recently a similar discussion in the nginx mailing list about 410 Gone and other error pages always excluded from gzip compression, and I think the sentiment would equally apply here. (http://mailman.nginx.org/pipermail/nginx/2013-March/037890.html)

2
  • I use it to make the user wait: I send chunks every seconds, so that the user will wait for the redirection for X seconds without getting a timeout
    – Julien
    Mar 6, 2013 at 3:53
  • I'd advise you to first fix HTTP/1.0 to HTTP/1.1 (these things do make a difference), and make sure your chunked encoding is not improper. The newer version of nginx probably discards some headers on which you depend because they're wrong.
    – cnst
    Mar 7, 2013 at 3:00
1

I had the same issue streaming mp4 file through html5 video tag.

Safari and Firefox behaved normally whereas Chrome was triggering ERR_CONTENT_LENGTH_MISMATCH at some point (but it allowed me to wach several minutes of the video before failing).

The problem didn't reproduce after I turned off cache control for mp4 files.

1
  • Did you mean server cache? How did you disable it?
    – Tobia
    Feb 22 at 10:46
0

Sharing this answer I posted to SO in case it's helpful: https://stackoverflow.com/questions/50499637/mp4-video-safari-cloudflare-nginx-rails-no-play/59348509#59348509

I had a similar issue with mp4 playback due to chunks not being served, and confirmed the issue per Apple's guide, listed below. I verified I was downloading the whole file, and after the fix below, only the first chunk.

curl --range 0-99 http://example.com/test.mov -o /dev/null

I resolved my Safari .mp4 playback by changing my gzip compression settings in my nginx.conf, to remove gzip compression of .mp4 files.

Here's the block in nginx for reference. (Note: depending on how your app is configured, you may need to change the location line to location ~ \.mp4$ {

location ~ ^/(assets|system|videos)/  {
   expires max;
   add_header Cache-Control public;
   add_header ETag "";
   gzip on;
   gzip_http_version 1.1;
   gzip_vary on;
   gzip_comp_level 6;
   gzip_proxied any;

   # Reference configuration
   #gzip_types text/plain text/html text/css application/json application/javascript application/x-javascript text/javascript video/mp4 application/mp4 image/jpeg image/png image/svg+xml application/x-font-ttf application/x-font-truetype application/font-woff application/font-woff2 application/vnd.ms-fontobject;

   # Kelton trying to fix cloudflare by removing the mp4 settings
   gzip_types text/plain text/html text/css application/json application/javascript application/x-javascript text/javascript image/jpeg image/png image/svg+xml application/application/x-font-ttf application/x-font-truetype application/font-woff application/font-woff2 application/vnd.ms-fontobject;
}

Link to Apple documentation reference: https://developer.apple.com/library/archive/documentation/AppleApplications/Reference/SafariWebContent/CreatingVideoforSafarioniPhone/CreatingVideoforSafarioniPhone.html#//apple_ref/doc/uid/TP40006514-SW6

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .