Nginx and Gunicorn 'err_conn_refused', but nginx not logging error

Question

I've been playing around with Django Cookie Cutter for local projects, and wanted to put it on my DO Droplet as a semi-production staging environment.

The server is set up with Nginx reverse proxying to a variety of projects, some served with Gunicorn, one with bare Flask, and one with uWSGI. Each project has its own virtualenv.

On deploying the project and setting it up, I have the following succession of errors.

Nginx conf is as follows (which mirrors the other sites on this droplet):

server {
        server_name test.myserver.com;

        access_log off;

        location /static/ {
            alias /var/www/myproject/static/;
        }

        location / {
                proxy_pass http://127.0.0.1:8085;
                proxy_set_header X-Forwarded-Host $server_name;
                proxy_set_header X-Real-IP $remote_addr;
                add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
        }
    }

In the first instance, after resetting nginx and not setting up Gunicorn, this 'works'/does what I expect: throws a 502 Bad Gateway error and the nginx logs note:

2016/02/25 11:27:25 [error] 21217#0: *275 connect() failed (111: Connection refused) while connecting to upstream, client: my.ip.address, server: test.myserver.com, request: "GET / HTTP/1.1", upstream:$

Fine, so, starting Gunicorn:

/var/www/test.myserver.com/env/bin/gunicorn -b 127.0.0.1:8085 myproject.wsgi:application &

(run from folder that has manage.py in) - this throws a 400 Bad Request in the browser; the gunicorn server notes (while I'm ssh'd in and can see):

[Django] ERROR: Invalid HTTP_HOST header: u'127.0.0.1:8085'.You may need to add u'127.0.0.1' to ALLOWED_HOSTS.

Makes sense: django-cookiecutter puts most of the sensitive config in environment variables. My .env file (and I've checked that it's available in shell) looks like this:

TERM=xterm-256color
SHELL=/bin/bash
SSH_CLIENT=my.ip.address 57044 22
OLDPWD=/var/www/myproject/env
SSH_TTY=/dev/pts/0
USER=root
VIRTUAL_ENV=/var/www/myproject/env
MAIL=/var/mail/root


PATH=/var/www/treatout/env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
        PWD=/var/www/treatout/env/bin
        LANG=en_US.UTF-8
        PS1=(env)${debian_chroot:+($debian_chroot)}\u@\h:\w\$ 
        SHLVL=1
        HOME=/root
        LOGNAME=root
        SSH_CONNECTION=82.23.204.106 57044 178.62.47.176 22
        _=/usr/bin/env

If I export DJANGO_ALLOWED_HOSTS=127.0.0.1 (the env variable which maps to ALLOWED_HOSTS in Django's Settings) or anything else (string, not string, wildcard, list), it seems to completely break something, and I get a 'ERR_CONN_REF' error in the browser, and the browser URL switches from test.myserver.com to 127.0.0.1:8085 (so obviously it doesn't connect, i'm not running a server on my localhost at that port.)

There's nothing in the Nginx Log (i'd expect an upstream error here, basically), and Gunicorn never sees the request. It doesn't (weirdly) seem to show in the nginx access log either.

I've also tried renaming the env.example that contains all my variables to .env, but it doesn't seem to make much difference.

I've obviously missed something, but not sure what, and struggling to debug; the other sites running are fine, and nginx generally is unaffacted. Trying unset DJANGO_ALLOWED_HOSTS has removed it from .env, but I still have the problem, after resetting nginx.

Edit: I got back to the last 'working' config (i.e. throwing the 400 Bad Request and things being logged) only deleting and rebuilding the virtualenv; I don't really want to do that multiple times as there are a lot of dependencies and it takes quite a while!

Second edit: On inspecting the environment settings using django-environ it looks like that export command has messed up the format:

env('DJANGO_ALLOWED_HOSTS')
'my.server.ip.address:127.0.0.1'

That doesn't look right, shouldn't it be comma separated? I feel like that was the right command to enter before, so reasonably unsure how to proceed here...

Answer 1

The answer in this particular case related to the lack of a join between the production.py settings file and the environment settings, when you don't use Heroku or Docker. (Taken from Issue 490)

My problems were twofold here: wrong format for the allowed hosts, caused by improperly concatenating the strings, and needing to put:

"In your settings file (e.g. common.py) you can read it like so: env.read_env(ROOT_DIR('.env')) depending on where you put your copy."

Doing so immediately made the (previously set) environment variables available to both shell and the program. I won't go into the knock-on bugs that I had as a result of endless faffing, those are another story. :)

Edit after a while: There was a related issue to this - in the nginx conf file for the site (in sites-available) I had to add

proxy_set_header Host $http_host; to ensure that the correct hostname was passed; this prevents the redirect to localhost that happens in certain circumstances.

My entire site file is therefore now:

server {
        server_name myservernamehere.com;

        access_log off;

        location /static/ {

            alias /var/www/mysitename/mysitename/mysitename/static/;
        }

        location / {
                proxy_pass http://127.0.0.1:8085;
                client_max_body_size 20M;
                proxy_set_header Host $http_host;
                proxy_set_header X-Forwarded-Host $server_name;
                proxy_set_header X-Real-IP $remote_addr;
                add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
        }
    }