nginx: WSS Load Balancer: Failed server not taken out of rotation, why?

Question

Using the current nginx, 1.6.2 to load balance wss (websockets) to two servers. We use some stickyness, as you see below.

This config works great, until one of the upstream servers goes off line.

The issue: When one of the upstream servers goes off line, nginx sees it is off line, but continues to try to use it every 8 to 12 seconds. We have tried adjusting fail_timeout, but it makes no difference. Even if we set it to 100, it behaves as if it is set to 5 or 10. Note: We are balancing websocket traffic.

Result: For several seconds, all requests go to the remaining on line server, but then every appx 10 seconds, it attempts to connect to the down server. This causes a significant (400-1000ms pause) for one or more clients.

We need to be able to take a server off line (for an upgrade, etc), and have nginx avoid that server until we bring it back.

We have also tried marking the upstream server as down, reloading the conf, and the behavior is still as described.

We are starting to guess that some of these features perhaps are not working as expected with websockets?

What is the solution?

worker_processes  1;    

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;

    keepalive_timeout  65;

    upstream websocketserver {
      hash $request_uri;
      server 192.168.97.102:3842 max_fails=1 fail_timeout=5s;
      server 192.168.97.202:3842 max_fails=1 fail_timeout=5s;
    }

    server {
        listen       127.0.0.1:3841;
        listen       127.0.0.1:3843 ssl;

        ssl_certificate      cert.cer;
        ssl_certificate_key  cert.key;

        ssl_protocols SSLv3 TLSv1;
        ssl_ciphers HIGH:!aNULL:!MD5;

        location / {

          proxy_pass http://websocketserver;

          proxy_next_upstream     error timeout invalid_header http_500;
          proxy_connect_timeout   2;
          proxy_read_timeout      86400;

          # WebSocket support (nginx 1.4)
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "upgrade";    
        }
    }
}

Answer 1

Answer was found.

Problem setup: 1.6.x running on Windows (this was for prototyping, not production)

Works Properly setup: 1.7.9 on Centos

We suspect the problem is that on Windows, some modules or such are not working properly.