Haproxy sticky session not redirecting to different server when backend fails

Question

I have the following configuration for my haproxy LB:

global
    daemon
    maxconn 2048

    # SSL
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private
    ssl-default-bind-ciphers ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM;

defaults
    log global
    mode    http
    option forwardfor

# handle incoming requests to port 80 (http)
frontend www-http
    bind 1.2.3.4:80
    reqadd X-Forwarded-Proto:\ http
    default_backend www-backend

# handle incoming requests to port 443 (https)
frontend www-https
    bind 1.2.3.4:443 ssl crt /etc/ssl/private/example.com.pem
    reqadd X-Forwarded-Proto:\ https
    default_backend www-backend

backend www-backend
   # always use https
   redirect scheme https if !{ ssl_fc }

   # RR algorithm for load balancing
   balance roundrobin
   option httpclose

   # tracke which backend served specific user
   cookie _rails_srv insert

   # sticky sessions
   appsession _rails_session len 64 timeout 24h
   server s1 4.5.6.7:80 check cookie s1
   server s2 7.8.9.0:80 check cookie s2

It's tied to 2 Rails application servers in the backend, and I'm using the session cookie provided by Rails (_rails_session) for session stickiness.

It works great until one of the servers goes down, and then clients with existing session to the failed server that are trying to access that server are getting 500 server error responses instead of being redirected to the other functioning backend.

I figured Haproxy will automatically redirect traffic to the other server when detecting failure. Am I doing something wrong in the configuration? thanks.

Answer 1

You are missing the redispatch option, apparently.

From the documentation:

option redispatch / no option redispatch: Enable or disable session redistribution in case of connection failure

In HTTP mode, if a server designated by a cookie is down, clients may definitely stick to it because they cannot flush the cookie, so they will not be able to access the service anymore.

Specifying "option redispatch" will allow the proxy to break their persistence and redistribute them to a working server.