4

I have the following NGINX config file:

worker_processes 4;
worker_rlimit_nofile 40000;

events {
    worker_connections 8192;
}

stream {

    upstream rancher_servers_http {
        least_conn;
        server <IP_NODE_1>:80 max_fails=3 fail_timeout=5s;
        server <IP_NODE_2>:80 max_fails=3 fail_timeout=5s;
        server <IP_NODE_3>:80 max_fails=3 fail_timeout=5s;
    }

    server {
        listen     80;
        proxy_pass rancher_servers_http;
    }

    upstream rancher_servers_https {
        least_conn;
        <IP_NODE_1>:443 max_fails=3 fail_timeout=5s;
        <IP_NODE_2>:443 max_fails=3 fail_timeout=5s;
        <IP_NODE_3>:443 max_fails=3 fail_timeout=5s;
    }

    server {
        listen     443;
        proxy_pass rancher_servers_https;
    }
}

And I would like to whitelist certain IPs for the subdomain *.dev.mydomain.com.

I tried to add this to the server block:

if ($host ~ *.dev.mydomain.com) {
    allow: <ip1>,<ip2>
    deny: all;
}

But I have the following error:

nginx: [emerg] "if" directive is not allowed here

I then tried to add a map. Under the server block I also have a directive not allowed here but I can add the map inside the stream block.

When I add the map on the stream block as follow:

map $hostname $deny_ips {
    default all;
    ~*.dev.mydomain.com all;
}

I can't use the deny_ips variable in server block, I get a

[emerg] invalid parameter "deny_ips" in /etc/nginx/nginx.conf:35

Note also that I can't map $host but I can only map $hostname or I got a

nginx: [emerg] unknown "host" variable

Can someone please help me to whitelist certain ips based on the subdomain ?

Thank you.

  • Just noticed the stream part; what are you trying to load-balance? If it's just HTTP/S traffic you should be able to use http instead of stream. – Ionut Ticus May 13 at 12:38
  • I'm loadbalancing nodes on a kubernetes cluster (built with rancher rke). I'm not 100% sure but I think I should use a TCP load-balancing in that case, as requests on pods might be on a different protocol than http. It's not actually the case but it might be. – MHogge May 13 at 14:15
  • The thing is that for a simple TCP connection there's no "server_name" variable that nginx can interpret and act upon. But if you're using nginx to expose services from your kubernetes cluster I'm guessing you will only expose HTTP/S services so you can give the solution below a try. – Ionut Ticus May 13 at 14:42
  • I think the pods themselves will communicate with one another directly (rather than via nginx) so you should be OK there (at least that's how it is on docker swarm which I used). – Ionut Ticus May 13 at 14:43
  • Ok I could give it a try but would I be able to keep the SSL cert management on the kubernetes cluster as it is ? I'm new to nginx and I often see SSL management on the nginx part when it's about http. That was also one of the reason why I preffered tcp load balancing. – MHogge May 14 at 7:26
3

I tried to find a solution with nginx but did not get far; I used an http block for regular HTTP traffic (this parts works OK) and a stream block for SSL (see second part of the answer for details).

However, I think haproxy can be used here as it seems to be more flexible when configuring ACLs.

haproxy solution

global
    maxconn     20000
    log         127.0.0.1 local0
    user        haproxy
    chroot      /usr/share/haproxy
    pidfile     /run/haproxy.pid
    daemon

defaults
    timeout connect 10s
    timeout client 30s
    timeout server 30s

frontend ssl
  bind *:443
  mode tcp
  acl network_allowed src 192.168.1.0/24 192.168.2.3/32  <-- EDIT IPs
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }
  acl dev_domain req_ssl_sni -m end .dev.mydomain.com  <-- EDIT domain
  tcp-request content reject if !network_allowed dev_domain
  use_backend rancher_servers_https

backend rancher_servers_https
  balance leastconn
  mode tcp
  server node1 1.2.3.4:443 fall 3 rise 2  <-- EDIT IP
  server node2 2.3.4.5:443 fall 3 rise 2  <-- EDIT IP
  server node3 3.4.5.6:443 fall 3 rise 2  <-- EDIT IP

I marked the place where you would need to edit the config and you'll probably need to tweak the global section as well (depending on your distro).
The above is for the SSL part; the non-encrypted part should be similar but without the SNI/SSL parts. Please note I haven't tried this and I have not used haproxy before but I think it will set you on the right path.

nginx solution - incomplete

I used map $ssl_preread_server_name for getting the domain name via SNI and a geo block for matching non-allowed IPs.

map $ssl_preread_server_name $dev_upstream {
    hostnames;
    *.dev.mydomain.com  1;
    default     0;
}

geo $not_dev_whitelist {
    default         1;
    192.168.1.0/24  0;
    2001:0db8::/32  0;
}

However because if is not allowed in the stream block I can't do:

server {
    listen     443;
    if ($dev_upstream) {
        set $dev "1";
    }
    if ($not_dev_whitelist) {
        set $dev "${dev}1";
    }
    if ($dev = "11") {
        return "blocked";
    }
    proxy_pass rancher_servers_https;
}
| improve this answer | |
3

First Nginx does not allow nested if statements. You need to use map functionality to achieve your goal.

Second you are wrong with map implement

map "$http_referer:$arg_78up" $refernotok {
    default 1;
    ~ mywebsite.com.*:.*$ 0;
    ~ ^.+:.+$ 0;
}

if ($refernotok = "1") {
    return 404;
}

you should use $http_referer for your map .

First part is regular expression. ^ means start of string, .+ means one or more characters, : is a delimiter character, and then again we match one or more characters with .+, and finally we require the string ending with $. 0 is the value $refernotok variable gets when the regular expression matches.

| improve this answer | |
  • How is HTTP referer relevant in this case? It's a stream balancer, not an HTTP balancer. – Ionut Ticus May 20 at 8:20
0

Just use two server blocks.

    server {
        listen       80  default_server;
        server_name  _;

        proxy_pass rancher_servers_http;
    }

    server {
        listen       80;
        server_name  *.dev.mydomain.com;

        allow <ip1>;
        allow <ip2>;
        deny all;

        proxy_pass rancher_servers_http;
    }

.. same for the server on :443

Also. if you're using nginx-ingress you can whitelist IPs with annotations. See docs. I find it easier to manage than manually configuing nginx.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.