whitelisting IP in NGINX stream directive

Question

I have the following NGINX config file:

worker_processes 4;
worker_rlimit_nofile 40000;

events {
    worker_connections 8192;
}

stream {

    upstream rancher_servers_http {
        least_conn;
        server <IP_NODE_1>:80 max_fails=3 fail_timeout=5s;
        server <IP_NODE_2>:80 max_fails=3 fail_timeout=5s;
        server <IP_NODE_3>:80 max_fails=3 fail_timeout=5s;
    }

    server {
        listen     80;
        proxy_pass rancher_servers_http;
    }

    upstream rancher_servers_https {
        least_conn;
        <IP_NODE_1>:443 max_fails=3 fail_timeout=5s;
        <IP_NODE_2>:443 max_fails=3 fail_timeout=5s;
        <IP_NODE_3>:443 max_fails=3 fail_timeout=5s;
    }

    server {
        listen     443;
        proxy_pass rancher_servers_https;
    }
}

And I would like to whitelist certain IPs for the subdomain *.dev.mydomain.com.

I tried to add this to the server block:

if ($host ~ *.dev.mydomain.com) {
    allow: <ip1>,<ip2>
    deny: all;
}

But I have the following error:

nginx: [emerg] "if" directive is not allowed here

I then tried to add a map. Under the server block I also have a directive not allowed here but I can add the map inside the stream block.

When I add the map on the stream block as follow:

map $hostname $deny_ips {
    default all;
    ~*.dev.mydomain.com all;
}

I can't use the deny_ips variable in server block, I get a

[emerg] invalid parameter "deny_ips" in /etc/nginx/nginx.conf:35

Note also that I can't map $host but I can only map $hostname or I got a

nginx: [emerg] unknown "host" variable

Can someone please help me to whitelist certain ips based on the subdomain ?

Thank you.

Answer 1

I tried to find a solution with nginx but did not get far; I used an http block for regular HTTP traffic (this parts works OK) and a stream block for SSL (see second part of the answer for details).

However, I think haproxy can be used here as it seems to be more flexible when configuring ACLs.

haproxy solution

global
    maxconn     20000
    log         127.0.0.1 local0
    user        haproxy
    chroot      /usr/share/haproxy
    pidfile     /run/haproxy.pid
    daemon

defaults
    timeout connect 10s
    timeout client 30s
    timeout server 30s

frontend ssl
  bind *:443
  mode tcp
  acl network_allowed src 192.168.1.0/24 192.168.2.3/32  <-- EDIT IPs
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }
  acl dev_domain req_ssl_sni -m end .dev.mydomain.com  <-- EDIT domain
  tcp-request content reject if !network_allowed dev_domain
  use_backend rancher_servers_https

backend rancher_servers_https
  balance leastconn
  mode tcp
  server node1 1.2.3.4:443 fall 3 rise 2  <-- EDIT IP
  server node2 2.3.4.5:443 fall 3 rise 2  <-- EDIT IP
  server node3 3.4.5.6:443 fall 3 rise 2  <-- EDIT IP

I marked the place where you would need to edit the config and you'll probably need to tweak the global section as well (depending on your distro).
The above is for the SSL part; the non-encrypted part should be similar but without the SNI/SSL parts. Please note I haven't tried this and I have not used haproxy before but I think it will set you on the right path.

nginx solution - incomplete

I used map $ssl_preread_server_name for getting the domain name via SNI and a geo block for matching non-allowed IPs.

map $ssl_preread_server_name $dev_upstream {
    hostnames;
    *.dev.mydomain.com  1;
    default     0;
}

geo $not_dev_whitelist {
    default         1;
    192.168.1.0/24  0;
    2001:0db8::/32  0;
}

However because if is not allowed in the stream block I can't do:

server {
    listen     443;
    if ($dev_upstream) {
        set $dev "1";
    }
    if ($not_dev_whitelist) {
        set $dev "${dev}1";
    }
    if ($dev = "11") {
        return "blocked";
    }
    proxy_pass rancher_servers_https;
}

Answer 2

First Nginx does not allow nested if statements. You need to use map functionality to achieve your goal.

Second you are wrong with map implement

map "$http_referer:$arg_78up" $refernotok {
    default 1;
    ~ mywebsite.com.*:.*$ 0;
    ~ ^.+:.+$ 0;
}

if ($refernotok = "1") {
    return 404;
}

you should use $http_referer for your map .

First part is regular expression. ^ means start of string, .+ means one or more characters, : is a delimiter character, and then again we match one or more characters with .+, and finally we require the string ending with $. 0 is the value $refernotok variable gets when the regular expression matches.

Answer 3

Just use two server blocks.

    server {
        listen       80  default_server;
        server_name  _;

        proxy_pass rancher_servers_http;
    }

    server {
        listen       80;
        server_name  *.dev.mydomain.com;

        allow <ip1>;
        allow <ip2>;
        deny all;

        proxy_pass rancher_servers_http;
    }

.. same for the server on :443

Also. if you're using nginx-ingress you can whitelist IPs with annotations. See docs. I find it easier to manage than manually configuing nginx.