1

Overview

I have an Nginx reverse proxy setup pointing to a SaaS application (BigCommerce). While my configuration works great, I'm not able to ensure the client IP shows in the SaaS backend instead of the Reverse Proxy IP. In the SaaS backend, there's no mechanism to add a list of trusted IP addresses or use set_real_ip_from and real_ip_header so instead I've been tasked to implement proxy_protocol on the Reverse Proxy in order to ensure IP headers are passed through the Reverse Proxy with the client's IP instead of its own.

Configuration

In http server context

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $host:$server_port;

    # support http 1.1 persistent connections
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

Issue

When I enable proxy_protocol on my listen directive within the http server context (e.g. listen 443 ssl http2 proxy_protocol), I receive a Broken Header error in nginx and a connection_reset in my browser.

Additonally, I've tried enabling proxy_protocol via the stream context as it's specified in "Accepting Proxy Protocol" docs by Nginx:

In stream context

stream {
    server {
        listen 12345;
        proxy_pass example.com:12345;
        proxy_protocol on;
    }
}

I'm unfamiliar with proxy_protocol so I don't know what I'm missing to get this working.

Technically speaking, I'm really trying to make my reverse proxy a 'transparent proxy' but it doesn't seem to work that way with the SaaS Backend because it's logging the proxy IP instead of the client IP.

2

You are looking in the wrong direction:

  1. The listen 443 ssh http2 proxy_protocol; directive (cf. listen) enables the parsing of the Proxy Protocol on incoming connections. Your browser does not speak the protocol, hence the Bad Header error. If you want to see a minimal example of Proxy Protocol use:

    openssl s_client -connect <nginx_ip>:https -crlf
    

    and type:

    PROXY TCP4 10.0.0.1 10.0.0.1 443 443
    GET / HTTP/1.1
    Host: <nginx_host_name>
    

    followed by two blank lines.

  2. The proxy_protocol on; directive (cf. proxy_protocol) would enable the Proxy Protocol between Nginx and the SaaS server. The latter, however, does not support the Proxy Protocol as you say. You'll need to implement it.

However, you have a third solution, which works if nginx is on the path between the SaaS server and the Internet:

  1. Use the proxy_bind directive (cf. proxy_bind) to tell nginx to spoof its source address (to use the one from the client):

    proxy_bind $remote_addr transparent;
    
  2. nginx can easily spoof the source address, but the response packet from the SaaS server will be routed to the client instead of nginx. Therefore you need to intercept the packets coming from the SaaS server. You need a new routing table, let's call it transparent by adding:

    100 transparent
    

    to /etc/iproute2/rt_tables. Then you need to accept any address as local:

    ip route add local default dev lo table transparent
    

    finally you need to add a firewall rule to force the usage of the routing table transparent on all traffic coming back from the SaaS server (let's assume its address is 10.10.10.10):

    iptable -t mangle -A PREROUTING -p tcp -s 10.10.10.10 --sport 80 -m socket\
    --transparent -j MARK --set-mark 1
    ip rule add fwmark 1 lookup transparent
    
7
  • Thank you for your response! If I were to use proxy_protocol on; within a stream context, do you know how I'd configure that so those headers are passed along with any requests like client -> reverse_proxy (with proxy_protocol) -> BigCommerce (SaaS Server)? The problem I may have in solution 3, if I'm understanding correctly, is that if I don't know the IP for the SaaS server I wouldn't be able to catch the traffic specifically from that server?
    – cvanorman
    Feb 28 '20 at 23:39
  • If nginx is not a router between the clients and the server, you won't intercept the traffic. The X-Forwarded-For headers are already sent to the server without activating the Proxy Protocol or anything else. The question is, whether the server interprets them. Feb 28 '20 at 23:45
  • To elaborate, I'm being told to implement proxy_protocol on the nginx Reverse Proxy so that IP addresses are forwarded correctly to the BigCommerce (SaaS Server). My proxy_pass is passing requests to store-xxxxxx.mybigcommerce.com and my x-forwarded-for headers are intact with the client_ip, proxy_ip but they're stating my x-real-ip header needs to be the client's IP which is where I'm having this issue.
    – cvanorman
    Feb 28 '20 at 23:46
  • The X-Real-IP is already set to the correct value. You are probably testing from behind the same NAT. Feb 28 '20 at 23:49
  • Ah, that makes more sense. So it's possible my host is not forwarding the correct value and setting $remote_addr to the proxy IP? Am I understanding you correctly?
    – cvanorman
    Feb 28 '20 at 23:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.