10

I have an nginx setup that receives requests from external hosts and proxies them to an internal server.

The config looks something like this:

server {

        listen 10.0.0.66:443;

        server_name my.example.com;

        root /websites/my.example.com

        ssl on;
        ssl_certificate /websites/ssl/my.example.com.crt;
        ssl_certificate /websites/ssl/my.example.com.key;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;

        location / {
                proxy_pass https://10.0.0.100:3000/;
        }
}

For experimental/testing purposes, I would like to be able to run what the internal host replied with through an arbitrary binary and respond with what the binary responds with.

For example, if I wanted to minify html on the proxy I would run the server's response through htmlcompressor and then send the output as the proxy's response to the client. The end result would be the end client getting minified html back.

I know there are all sorts of addons and examples for nginx to accomplish this for locally served data, but how to set it up for a proxy?

2
  • Just to clarify. You want nginx to forward the request to the proxied server, receive a response back, compress it, then forward it to the user? You want nginx to process it in the middle of the server and user?
    – sjdaws
    Feb 26, 2013 at 4:15
  • @sjdaws, not necessarily compress it, but run it through any arbitrary program and use the output as what gets sent to the client. So in essence yes, I want to modify the output going from the server to the client.
    – 0x6A75616E
    Feb 26, 2013 at 5:09

2 Answers 2

11
+50

So, you want nginx to proxy a request from the client to the backend server, and then, prior to returning the reply of the backend to the client, pipe such reply through another external processor?

I don't think you can do the above with any official nginx modules as provided by Igor Sysoev and Nginx, Inc currently. The closest thing that's available for altering the body of the response is a few filter modules that come together with nginx, but are turned off by default, including the add_before_body, add_after_body and sub_filter directives:

http://nginx.org/en/docs/http/ngx_http_addition_module.html
http://nginx.org/en/docs/http/ngx_http_sub_module.html

Also, perhaps gzip on; is what you actually want instead?

http://nginx.org/en/docs/http/ngx_http_gzip_module.html

Or, potentially, if you know perl and are willing to run an entirely experimental module, take a look at embedding perl into nginx, with an official nginx module that is turned off by default and is (somewhat obviously) totally experimental:

http://nginx.org/en/docs/http/ngx_http_perl_module.html

Another option is to use some kind of a Fast-CGI setup to which you'll be redirecting the requests, where, in turn, your Fast-CGI script will be doing requests to the backend, and then the final processing, prior to returning the replies back to nginx to cache and return to the user.

There is also proxy_set_body (but no fastcgi_set_body yet), to change the body of the request (e.g. from what the client has supplied), but there does not seem to be any equivalent directive or variable to get the body of the response, in order to pass to a somehow subsequent request to a post-processor. In any case, a filter module is probably what you want for a post-processor.

(Also, you do realise that a naive approach of forking and piping replies through a regular executive is going to be extra-extra slow, right?)

To summarise, I think gzip on; is exactly what you're looking for; else, provided you can modify the original webapp, I think your very best bet might be to install some kind of post-processor within the webapp itself, which would seem like the next easiest solution overall. Potentially, you could look into how the filter modules are implemented, e.g. the aforementioned ngx_http_addition_filter_module.c, plus some more obviously-relevant filters like ngx_http_gzip_filter_module.c, and implement your own embedded filter module. Or hire Nginx, Inc. to write this for you! But, seriously, gzip on; just works, and is likely to give you much better results without any hassles, performance or stability issues, and it's already compiled in by default, you just have to enable it in nginx.conf.

3
  • Thanks for your reply! I am aware of gzip on and what I'm trying to accomplish is more high level than deflating the output. I have a proxy that controls access to certain internal web services and I wanted to be able to append things like google analytics to the output, sort of like how cloudflare does it. Like you say, it sounds like fastcgi is an option, so I'll look into that. Thanks again!
    – 0x6A75616E
    Feb 28, 2013 at 22:35
  • If you just want to append stuff or add google analytics, then add_after_body or sub_filter is exactly what you need. The example at nginx.org/en/docs/http/ngx_http_sub_module.html shows exactly that scenario: replacing "</head>" with "</head><script …". You might have to recompile nginx in order to enable those modules (check nginx -V for how your nginx was compiled), but they're already standard modules otherwise.
    – cnst
    Feb 28, 2013 at 23:29
  • Have a look at subs_filter module too. Jan 28, 2014 at 11:06
1

I think if you want to run arbitrary code to mofify nginx output, ... You can write a lua script.

Look for "nginx lua".

(Example: http://www.londonlua.org/scripting_nginx_with_lua/slides.html?full#hello-lua)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .