64

I am using nginx/0.7.68, running on CentOS, with the following configuration:

server {
    listen       80;
    server_name ***;
    index index.html index.htm index.php default.html default.htm default.php;

    location / {
            root   /***;
            proxy_pass   http://***:8888;
            index  index.html index.htm;
    }
    # where *** is my variables

The proxy_pass is to a DNS record whose IP changes frequently. Nginx caches the outdated IP address, resulting in a request to the wrong IP address.

How can I stop nginx from caching the IP address, when it is outdated?

9

It's an intriguing question and AFAIK that's not going to work well. You can try to use the upstream module and use the directives for failover to see if it works as a hack.

2018 edit: a lot of things changed. Check the answer by @ohaal to get real information about this.

| improve this answer | |
  • 1
    surprisingly when i changed to upstream, everything worked as expected. I'll then mark this as correct answer – xiamx Feb 27 '11 at 22:20
  • 1
    According to the documentation, there's a special upstream server flag resolve that's only available in the commercial version (see nginx.org/en/docs/http/ngx_http_upstream_module.html#server) – omribahumi Jul 19 '15 at 13:48
  • 1
    @gansbrest that site seems to be some kind of spammy site? i'd ask that you remove your response. – majikman Jun 28 '18 at 0:02
  • It's a shame that in 2020 the upstream module still doesn't fix this issue. Resolving the address once and using that forever is useless. – Phil Sep 3 at 8:23
111

Accepted answer didn't work for me on nginx/1.4.2.

Using a variable in proxy_pass forces re-resolution of the DNS names because NGINX treats variables differently to static configuration. From the NGINX proxy_pass documentation:

Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.

For example:

server {
    ...
    resolver 127.0.0.1;
    set $backend "http://dynamic.example.com:80";
    proxy_pass $backend;
    ...
}

Note: A resolver (i.e. the name server to use) MUST be available and configured for this to work (and entries inside a /etc/hosts file won't be used in a lookup).

By default, version 1.1.9 or later versions of NGINX cache answers using the TTL value of a response and an optional valid parameter allows the cache time to be overridden:

resolver 127.0.0.1 [::1]:5353 valid=30s;

Before version 1.1.9, tuning of caching time was not possible, and nginx always cached answers for the duration of 5 minutes..

| improve this answer | |
  • wouldn't this force a dns query on every single request? that sounds like awful performance... – lucascaro Mar 10 '15 at 3:42
  • No, read the source. In such setup ip address of "foo.example.com" will be looked up dynamically and result will be cached for 5 minutes. I've added it to the answer for clarity. – ohaal Mar 10 '15 at 9:43
  • 17
    After spending most of my day on this - on Ubuntu 12.04 with nginx 1.1.19, set inside location doesn't work properly. Beware – omribahumi Jul 19 '15 at 13:38
  • This solution worked with me, however I couldn't find a reference for the 5 minutes TTL. nginx.org/en/docs/http/ngx_http_core_module.html#resolver By default, nginx caches answers using the TTL value of a response. An optional valid parameter allows overriding it: resolver 127.0.0.1 [::1]:5353 valid=30s; – Montaro Aug 15 '15 at 2:51
  • 6
    Note: for docker, it's DNS resolver resides at 127.0.0.11, so for development, I use this: resolver 127.0.0.11 [::1]:5353 valid=15s; – Dalibor Filus Jun 1 '18 at 11:18
9

There is valuable information in gansbrest comment and ohaal answer.

But I think it's important to mention this official nginx article, posted in 2016, it clearly explains nginx behaviour on this matter and the possible solutions: https://www.nginx.com/blog/dns-service-discovery-nginx-plus/

We indeed have to "Set the Domain Name in a Variable" and use the resolver directive.

however, using a variable changes the rewrite behaviour. You may have to use the rewrite directive, it depends on your location and proxy_pass setup.

PS: would have post a comment but not enough points yet...

| improve this answer | |
7

ohaal's answer takes most of us there, but there is a case where the DNS resolver does not live at 127.0.0.1 (eg when you're in a special containerized environment)

In that case, you may want to change the nginx conf to resolver ${DNS_SERVER};. Then, before you start nginx, run

export DNS_SERVER=$(cat /etc/resolv.conf |grep -i '^nameserver'|head -n1|cut -d ' ' -f2)
envsubst '${DNS_SERVER}' < your_nginx.conf.template > your_nginx.conf

Note, that you need the gettext package installed, as that provides the envsubst command.

| improve this answer | |
0

I've hacked together a script to watch a conf.d folder upstreams for dns changes and reload nginx upon detection. It's a first pass, and surely can be improved (next pass, I'll use nginx -T to parse upstreams specifically. Same idea could be used for proxy_pass directives):

#!/bin/bash

get_upstreams() {
  local files=$@
  grep -hEo '(server\s+)[^:;]+' $files | cut -d' ' -f 2
}

resolve_hosts() {
  local hosts=$@
  for h in $hosts; do dig +short $h; done | sort -u
}

watch_dir=$1

[ -d $watch_dir ] || exit 2

upstreams=$(get_upstreams $watch_dir/*)
ips=$(resolve_hosts $upstreams)
if [ ! "$ips" ]; then
  echo "Found no resolvable hosts in $watch_dir files."
fi

host_hash=$(echo $ips | /usr/bin/sha512sum)

echo $host_hash
echo $ips

while [ -d $watch_dir ]; do
  sleep 30
  upstreams=$(get_upstreams $watch_dir/*)
  ips=$(resolve_hosts $upstreams)
  new_hash=$(echo $ips | /usr/bin/sha512sum)
  if [ "$host_hash" != "$new_hash" ]; then
    echo Detected an upstream address change.  $ips
    echo Reloading nginx
    echo $new_hash
    echo $ips
    /sbin/service nginx reload
    host_hash=$new_hash
  fi
done
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.