I am following this tutorial to configure HAProxy and Let's Encrypt. The goal is to forward requests to example.com/sensu to a server instance listening on 127.0.0.1:3002. But when I visit example.com/sensu in my browser using http or https, I always get:

503 Service Unavailable

No server is available to handle this request.

Why can't I reach the server through HAProxy?

The setup is as following:

My domain has A record point to my router's public IP.

The server's host machine is behind the router and is assigned to a private IP (10.0.0.x). On the router, I forwarded all traffic on port 80, 443 to this private IP(10.0.0.x:80, 10.0.0.x:443) and successfully generated my certs using letsencrypt.

On the machine, I have this server, which is a uchiwa dashboard, running in a docker container and listening on 127.0.0.1:3002

$docker ps
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                                                                      NAMES
48889effb2fb        uchiwa/uchiwa        "/go/bin/uchiwa -c /c"   About an hour ago   Up About an hour    127.0.0.1:3002->3000/tcp                                                   uchiwa

and the port is reachable from the host machine:

$ telnet 127.0.0.1 3002
Trying 127.0.0.1...
Connected to 127.0.0.1.

HAProxy is directly running on the host. Here's my haproxy.cfg

global
        log /dev/log    local0
        log /dev/log    local1 notice
        debug
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        maxconn 500
        tune.ssl.default-dh-param 2048

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL). This list is from:
        #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3

defaults
        log     global
        mode    http
        option forwardfor
        option  httplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http
frontend www-http
   bind :80
   reqadd X-Forwarded-Proto:\ http
   acl uchiwa path_beg /sensu
   use_backend uchiwa-backend if uchiwa
frontend www-https
   bind :443  ssl crt /etc/haproxy/certs/example.com.pem
   reqadd X-Forwarded-Proto:\ https
   acl uchiwa path_beg /sensu
   acl letsencrypt-acl path_beg /.well-known/acme-challenge/
   use_backend uchiwa-backend if uchiwa
   use_backend letsencrypt-backend if letsencrypt-acl
backend uchiwa-backend
   redirect scheme https if !{ ssl_fc }
   server 127.0.0.1:3002 check
backend letsencrypt-backend
   server letsencrypt 127.0.0.1:54321

HAproxy log didn't show any anything after starting all the proxy services, even when it successfully redirected my http request to https:

Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy www-http started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy www-http started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy www-https started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy www-https started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy uchiwa-backend started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy uchiwa-backend started.
Jul 24 16:54:56 <hostmachine> haproxy[18640]: Proxy letsencrypt-backend started.
(nothing after this line...)

I also tried binding server to all interfaces(0.0.0.0:3002). Where did i go wrong? Please help.

==========================================================

UPDATE:

Turns out the server name is a required field. I updated my backend settings and now I don't see the 503 error anymore.

backend uchiwa-backend
   option forceclose
   redirect scheme https if !{ ssl_fc }
   server uchiwa 172.17.0.6:3000

However now I am getting 404 error.

404 page not found

And here's the log:

Aug  7 03:38:41 <hostmachine> haproxy[723]: 192.168.1.1:57720 [07/Aug/2016:03:38:41.957] www-https~ uchiwa-backend/uchiwa 4/0/0/0/5 404 195 - - ---- 0/0/0/0/0 0/0 "GET /sensu HTTP/1.1"
Aug  7 03:38:42 <hostmachine> haproxy[723]: 192.168.1.1:57721 [07/Aug/2016:03:38:42.293] www-https~ www-https/<NOSRV> -1/-1/-1/-1/3 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"
  • 1
    the edit makes the question much better. But please also include the log lines for the failed attempts; it is very likely that the answer is written there. – Jenny D Jul 24 '16 at 18:31
  • @JennyD Thank you. I just added the haproxy log. However I didn't see anything in the log, even when I had debug mode turned on. It didn't log anything when I saw it successfully redirected my http request to https. Is there anything else I need to do to configure HAProxy's logging handler? – Yuhao Zhang Jul 24 '16 at 22:13

You are missing a server name in your unchiwa-backend:

server 127.0.0.1:3002 check

This should be something like:

server uchiwa 127.0.0.1:3002 check
  • Thank you! Didn't know server name is a required field. This solved one problem. But now I am getting "404 page not found". I don't think my current haproxy settings can handle 404 so I am assuming this message is from uchiwa. – Yuhao Zhang Aug 7 '16 at 7:39
  • Yes, this would be on the application level. HAProxy normally wouldn't serve 404's in a standard setup. – Mick Switser Aug 11 '16 at 14:45

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.