I'm new to using HAProxy to load balance my app, which currently runs five backend application instances. My setup is relatively straightforward. I have a Ruby app that uses fibers, EventMachine, and thin to try and keep it as non-blocking as possible. So most requests will return quickly, or at the very least they won't block the application server, so it can serve multiple requests at once.
However, there are some requests (such as image uploads which need to access shell commands) which can be slow and block the application server.
What I'm finding is that a straight-forward round-robin style of balancing doesn't make sense here, since requests which could be handled simultaneously and returned quickly get backlogged behind the slow requests. What's the best way to handle this?
Having a health-check which runs frequently (say every 250ms) to check and see if the server is responding. If it's not, assume it's "down" (most likely blocking on a long request), which will cause HAProxy to route requests around it. However in this scenario there is a possibility that all 5 instances could become blocked.
Have a pre-defined list of slow request URLs, designate 2 or 3 of the application backends to only handle slow requests, and route all others to the "fast" backends. In theory fast requests will never get blocked, but this approach seems a bit more brittle since I'll need to make sure that if the URL's ever change, I remember to update my HAProxy config.
I think the latter approach is probably best, but since devops isn't my strong-suit, I thought I would check and see what best practice is in this scenario.
Of course, "best practice" is probably to have all long-running requests moved to background tasks, but in this case let's assume I don't have time for that right now if it's avoidable :-)