2

Problem

Suppose we have a compressed archive on disk, e.g. file.tar.gz, which should be served as-is.

The file is served with Content-Type: application/gzip, but for some reason the server also adds a Content-Encoding: gzip header, even though it does not apply any additional on-the-fly compression (so no mod_deflate or similar).

As a result, some user-agents (e.g. Python's requests and urllib3) will automatically decompress the gzip file, so, after download, we end up with file.tar, instead of file.tar.gz.

Examples

One example is the Django development server, which adds a Content-Encoding header based on the filename, as can be seen here.

Another example is django-storages in combination with AWS S3. If a gzip file is uploaded to an S3 bucket using django-storages (v1.12.3), a Content-Encoding: gzip header is added to the file metadata, as can be seen here. When downloading that file from S3, the Content-Encoding from the metadata is added to the download response headers.

Question

Is this acceptable server behavior, or should the server be considered broken?

What I found

This excerpt from rfc7231 appears to suggest the behavior is not acceptable:

... Content-Encoding is primarily used to allow a representation's data to be compressed without losing the identity of its underlying media type. ... If the media type includes an inherent encoding, such as a data format that is always compressed, then that encoding would not be restated in Content-Encoding ...

although "primarily" implies there may be alternative uses.

Here's the same thing, in slightly different words, from Mozilla:

... If the original media is encoded in some way (e.g. a zip file) then this information would not be included in the Content-Encoding header.

However, rfc7231 then goes on to say the following, which (I think) suggests the behavior can be acceptable:

... Likewise, an origin server might choose to publish the same data as multiple representations that differ only in whether the coding is defined as part of Content-Type or Content-Encoding, since some user agents will behave differently in their handling of each response (e.g., open a "Save as ..." dialog instead of automatic decompression and rendering of content).

If this is acceptable server behavior, then I suppose our user-agent would need to take that into account.

1
  • See also ngx_http_gzip_static_module. Seems reasonable to mimic such logic in the dev server, given there is zero intersection between compression encodings and Django static file types.
    – anx
    Jul 21, 2022 at 17:14

0

You must log in to answer this question.

Browse other questions tagged .