2

I am running a site that uses Django, Nginx, Gunicorn, Supervisord and fail2ban (which only allows ssh, http and https). The site is live and working correctly but there are some nginx error log entries that are concerning:

connect() failed (111: Connection refused) while connecting to upstream, client: x.x.x.x, server: www.example.com, request: "GET /example/url/to/get/ HTTP/1.1", upstream: "http://[::1]:8000/example/url/to/get/", host: "www.example.com"

upstream server temporarily disabled while connecting to upstream, client: x.x.x.x, server: www.example.com, request: "GET /example/url/to/get/ HTTP/1.1", upstream: "http://[::1]:8000/example/url/to/get/", host: "www.example.com"

Here is my nginx config:

upstream app_server_wsgiapp {
  server localhost:8000 fail_timeout=0;
}

server {
 listen 80;
 server_name www.example.com;
 return 301 https://www.example.com$request_uri;
}

server {
  server_name           www.example.com;
  listen                443 ssl;

  if ($host = 'example.com') {
    return 301 https://www.example.com$request_uri;
  }

  ssl_certificate       /etc/nginx/example/example.crt;
  ssl_certificate_key   /etc/nginx/example/example.key;
  ssl_session_timeout   1d;
  ssl_session_cache     shared:SSL:50m;
  ssl_protocols         TLSv1.1 TLSv1.2;
  ssl_ciphers           'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-
AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-
SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-
SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-
SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-
SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-
AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-
SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-
SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-
SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-
SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-
CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
  ssl_prefer_server_ciphers   on;

  access_log          /var/log/nginx/www.example.com.access.log;
  error_log           /var/log/nginx/www.example.com.error.log info;
  keepalive_timeout   5;

  proxy_read_timeout    120s;

  # nginx serve up static and media files
  location /static {
    autoindex on;
    alias /static/path;
  }

  location /media {
        autoindex on;
        alias /media/path;
  }

  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    if (!-f $request_filename) {
        proxy_pass http://app_server_wsgiapp;
        break;
    }
  }
}

I do not have any errors in the Gunicorn logs.

Like I said, the site is working correctly. But I don't want to ignore error logs which could potentially become a bigger issue later.

8

Could this be because your system is dual-stack, but your upstream is IPv4 only?

It looks as if localhost is resolving to [::1], which depending on your upstream might be the problem in and of itself.

Given you are communicating over loopback, I would tend to assume the Connection refused is 'real' - it is reflective of the actual issue.

You can check whether this is the problem by replacing localhost with 127.0.0.1 in your upstream config:

upstream app_server_wsgiapp {
  server 127.0.0.1:8000 fail_timeout=0;
}
| improve this answer | |
  • Excellent, that appears to have fixed the issue. I unfortunately don't know enough about nginx to know why this fixed the issue, or why this was happening even with your explanation but im glad the issue is fixed. I will keep an eye on the logs for the rest of the day and then accept your answer once im sure it has been fixed. – darkpool Mar 28 '17 at 9:27
  • 1
    The upstream service is only listening on the IPv4 address of 127.0.0.1 and is not listening on the IPv6 address of ::1. – Hoylen Jul 27 '17 at 0:33
  • When Nginx tries to resolve "localhost" into an IP address, it first gets ::1, tries connecting to the upstream service on ::1 and fails, writing the error message. Since that failed, it tries the next IP address it can find for "localhost", which happens to be 127.0.0.1 which works so that result is returned to the client. If you look inside your /etc/hosts file, you'll see there are more than one IP address for the name of "localhost". – Hoylen Jul 27 '17 at 0:41
0

I was getting the following in nginx error.log which got resolved by replacing upstream localhost to 127.0.0.1 . For the same call I was getting a successful entry in access.log and a connection refused entry in error.log. Thanks so much for the help.

2017/08/15 16:41:27 [error] 17337#0: *382749 connect() failed (111: Connection refused) while connecting to upstream, client: 10.10.xx.xx, server: test.myhost.com, request: "GET /test/relaystate HTTP/1.1", upstream: "http://[::1]:8800/test/relaystate", host: "test.myhost.com"

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.