We are running multiple backend servers with fastcgi protocol and we balance between them using HAproxy. Here is a sample of config:

listen Balancer
    mode tcp
    option tcplog
    timeout connect 2000
    timeout server 2000
    timeout queue 2000
    timeout client 2000
    balance leastconn
    server backend1 check inter 2000 rise 2 fall 5
    server backend2 check inter 2000 rise 2 fall 5
    server backend3 check inter 2000 rise 2 fall 5
    server backend4 check inter 2000 rise 2 fall 5
    server backend5 check inter 2000 rise 2 fall 5
    server backend6 check inter 2000 rise 2 fall 5

The overall timeout is set to 2 seconds but most requests are handled under 0.3 seconds. The problem is that during the peaks sometimes some backend can not reply to a query under 2 seconds and then gateway timeout is returned.

What I would like to do is that when one server is selected (e.g. backend1) and it fails to deliver the reply in 1 second, HAproxy would select another backend and retry. If it fails again in 1 second then the timeout will take place.

So instead of waiting for one server for 2 seconds is it possible to wait 1 second for first, if it fails, try another and then fail?

  • 1
    AFAIK, you are asking to retry a different server when a request has been sent, but upstream takes too long to reply with a response. This a bad idea. Imagine you make a request that is really heavy (>5 seconds) to process for a server. If your haproxy then starts to send the same request to all other your upstream servers you 1) have no use of a haproxy loadbalancer 2) risk, in your case, increasing the load of all your upstream servers by 6. This is why haproxy doesn't have an basic option for it. I think you should consider optimizing your response times instead. – Ztyx Feb 18 '16 at 11:05
  • @Ztyx You have a good point, but what if I wanted to try a different server (in order to avoid returning an error) when the first one dies when processing the request? Say that requests are few and fast to process, but servers fail often and I absolutely want every request handled. – nphx Nov 8 '16 at 12:14
  • 1
    @nphx ...and you are fine with a single request being sent multiple times to your servers? That is, will all your requests be idempotent? It's definitely doable but a lot of work. I'm just wondering if servers are failing often enough in this way to spend time making all your requests idempotent. Feels like an edge case. – Ztyx Nov 8 '16 at 15:14

If I'm reading your question right, using setting timeout server to 1 second (1000 ms), and using option redispatch should give you the desired effect.

option redispatch
no option redispatch
Enable or disable session redistribution in case of connection failure In HTTP mode, if a server designated by a cookie is down, clients may definitely stick to it because they cannot flush the cookie, so they will not be able to access the service anymore.

Specifying "option redispatch" will allow the proxy to break their persistence and redistribute them to a working server.

It also allows to retry last connection to another server in case of multiple connection failures. Of course, it requires having "retries" set to a nonzero value.

This form is the preferred form, which replaces both the "redispatch" and "redisp" keywords.

If this option has been enabled in a "defaults" section, it can be disabled in a specific instance by prepending the "no" keyword before it.

  • 3
    If I understand the question correctly, the user j99 want to retry after a request has been sent to a server when a response takes took long. AFAIK, redispatch will only redispatch if the server can't be connected to on TCP level (or possibly if a full request can't be sent - unclear). – Ztyx Feb 18 '16 at 11:01

HAProxy 2.0 (2019-06-16) and newer support this if you configure retries and option redispatch. In your case you also need to tune retry-on accordingly.

The following blog article by HAProxy gives a great overview with examples: https://www.haproxy.com/blog/haproxy-layer-7-retries-and-chaos-engineering/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.