i have a setup with :

HAProxy acting as a layer 4 LB (just forwarding requests, passthrough, TCP mode for all frontend/backend) to multiple Apache frontends. Each of these apache frontends is sending the traffic to a tomcat backend where an application is running.

Now, if one of these tomcats/applications is crashed, the corresponding apache frontend reports an error 503 (which is normal), others are OK.

problem is : as HAProxy is just load balancing to each Apache randomly (based on source IP in my case), some users get the 503 errors (pointing to a tomcat not working), while others are working (pointing to a tomcat that is OK)

is there a way to detect that in any way ? (and ensure users are only redirected to a working frontend/backend)

i suppose i can't do anything on the HAproxy side as HAproxy is forwarding to the "Apache" backends (and reports them as healthy as port 443 is OK) and does not see the real backends (the tomcat servers). i suppose i have to do something on the apache frontends (such as "if there is a 503, stop apache" or something).

any idea on how to handle this ?

thanks !

ps : let me know if you want to see the config


You need to define a health check for your backends.

Find some simple request that tomcat can serve without incurring too much processing overhead; generally something like /img/empty.gif will be used. Point is that this can be checked to give a HTTP status 200 when tomcat is up and running, and some other error when tomcat is not working.

Modify your haproxy.conf:

listen app1
  mode http
  option httpchk GET /img/empty.gif HTTP/1.0
  http-check expect status 200
  server backend1 check fall 3 rise 2 ssl verify none
  server backend2 check fall 3 rise 2 ssl verify none

The thing you need to add is the option httpchk and http-check lines, I assume you already have the check in the server line.

If a server does not respond with status 200 when the httpchk is performed, then it will be taken out of the pool. It will be added back when it responds normally again.

| improve this answer | |
  • thank you, i was thinking in checking the tomcat listening port by adding a tcp check behind the server like "server backend1 check tcp 2222", but your solution seems good, i can do a checkpage or status page and check the status that's great idea, will have to exempt it from SSO though as i forgot to mention that the webserver also has SSO in between. the problem is my setup is full TCP (cannot use HTTP mode) so i don't think i can do http checks in TCP mode.. – olivierg Nov 8 '19 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.