3

Summary - need to add whitelist ips to the mysite1.example.com. now when they are added it doesn't work as every request is coming as originated from the load balancer server.

Im working on a setup with a front seating Nginx host with a upstream backend to loadbalance all tcp packets on port 443 to backend servers.

nginx config of Loadbalancer server running nginx - server C is as below

stream {
        upstream stream_backend {
                hash $remote_addr consistent;
                server 10.15.15.3:443;   ## server A
                server 10.15.15.9:443;   ## server B
        }


        server {
                listen     443;
                proxy_pass stream_backend;
                proxy_timeout 5s;
                proxy_connect_timeout 5s;
        }
}

server A and server B has below nginx.conf. they are identical servers with apps.

it has two virtual hosts running in each. they are working fine.

http {

    server {
        server_name mysite1.example.com;
        listen *:443 ssl;
        listen [::]:443 ssl;
        
        allow 123.45.85.220; # this seems not working
        deny all; # only this is working

        
        location ^~ /static/ {
            ...
        }
        ...
        
        ssl_certificate        file.pem;
        ssl_certificate_key    file.key;
    }


    server {
        server_name mysite2.example.com;
        listen *:443 ssl;
        listen [::]:443 ssl;
        
        
        location /somethin {
            ...
        }
        location /something2{
            ...
        }
        
        ssl_certificate        file.pem;
        ssl_certificate_key    file.key;
    }
}

what I need is to whitelist only few ips to the virtual host mysite1.example.com. the issue I face is that the nginx running on Server A and B see the load balancer Ip as the client Ip. so when tried adding allow IP; deny all. doesn't work for any host as it has the load balancer IP on all requests as the client IP.

Can someone guide me on adding proxy IP configs to achieve the above mentioned setup running fine. Setup is complete except for the IP whitelist issue.

p.s SSl termination happens at the back-end servers , server A and Server B

I've searched through the web and found these helpful but still couldn't figure out how to get it all working.

https://stackoverflow.com/questions/40873393/nginx-real-client-ip-to-tcp-stream-backend https://www.cyberciti.biz/faq/nginx-redirect-backend-traffic-based-upon-client-ip-address/

3
  • 1
    Its really Hard to understand the original request - can be be more precise and format your Question to be easier to read?
    – djdomi
    Jun 14, 2021 at 7:50
  • added the summary and comments on the config. need to find a way to whitelist ips and deny all other ips in mysite.example.com
    – Laksith
    Jun 14, 2021 at 8:50
  • the easiet way to restrict access is the use of allow and deny, but in order first deny and then allow
    – djdomi
    Jun 14, 2021 at 9:39

3 Answers 3

5

This is happening because your nginx load balancer makes a new TCP connection to your nginx web servers, causing the original client IP address to be lost.

You can solve this using the PROXY protocol. This protocol sends the original IP address information when the new connection is opened so that your web servers can be aware of it.

To set it up, you need to make the following changes:

  • On the load balancer, set proxy_protocol on;:

    stream {
        server {
            proxy_pass somewhere;
            proxy_protocol on;
    
  • On the web server, accept the PROXY protocol, and enable the Real IP functionality for the load balancer's IP address.

    server {
        listen 443 ssl http2 proxy_protocol;
        listen [::]:443 ssl http2 proxy_protocol;
        real_ip_header proxy_protocol;
        set_real_ip_from <load balancer's IP>;
    

There are additional tweaks you can make to the PROXY protocol which you can find in the nginx documentation but this should get you started and solve the immediate problem.

4

My approach would be:

Filter in the load balancer using SNI:

Basically: SNI allows the proxy to see the domain name, without terminating TLS. Here is a guide. And here are the nginx docs to combine it into a solution. If you can do this, you can then deny based upon $ssl_preread_server_name.

if ( $ssl_preread_server_name ~* mysite1.example.com && $remote_addr !~* <whitelisted_ip> ) {
           return 404;

The real_ip module also seems like a good solution.

Edit

The PROXY protocol would actually be better than real_ip. But this would still mean, that your backend servers get hit by requests that they'd end up blocking. The SNI solution catches them at the load-balancer.

1
  • Relaying on the SNI is unreliable, as browsers are supporting ESNI and ECH extensions
    – Ferrybig
    Jun 14, 2021 at 17:54
0

seems like your proxy not passing the real IP try to add this parameter

proxy_set_header X-Real-IP $remote_addr;

can you check the access logs which ip there ?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .