Rate limiting HAProxy connections from clients behind another proxy or NAT

Question

We have HAProxy running on our pfSense hardware, forwarding a single frontend to a number of backend services (using cookies). Everything is working great.

We'd like to add basic "rate limiting" to HAProxy. In the Frontend configuration, I've added

stick-table  type ip  size 100k  expire 20s  store http_req_rate(10s)
http-request track-sc0 src 
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }

However, one of our customers has several hundred users at the same site, and they're connecting to one of our applications (which is quite "chatty" in network terms) from behind a proxy server - therefore, the requests that hit HAProxy are all from the same source IP. The users all tend to log into the application at the same time of day, so I'm wary of setting a limit on the number of new connections.

Whilst I'm conscious this might be a "no win" situation, is there any advice on how to handle this scenario? I could, of course, raise the rate limit well above the maximum traffic level, but it strikes me that this lessens the usefulness of any rate limiting in preventing malicious attacks.

Answer 1

I guess you're only options are to use cookies or possibly (but unlikely) the x-forwarded-for header. The cookie option would be similar to black Friday protection: http://www.loadbalancer.org/blog/black-friday-black-out-protection-with-haproxy/ It would be very interesting to know if you could get a working example for the problem that you face.

Answer 2

You may use the X-Forwarded-For header instead of your source IP to allow your "special" customer more requests. For example:

    stick-table  type ip  size 100k  expire 30s  store http_req_rate(10s)
    http-request track-sc0 hdr(X-Forwarded-For)
    acl vip_ip hdr_ip(X-Forwarded-For) 1.2.3.4
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 } !vip_ip
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 500 } vip_ip

Here, anyone with X-Forwarded-For HTTP Header with IP matching 1.2.3.4 will be allowed 500 requests per 10 seconds, instead of the normal 100.

Please note that the X-Forwarded-For header can be spoofed, so it's not a completely reliable method for access control. Perhaps your WAF (proxy server) can limit clients with X-Forwarded-For, and deny access if possible.