Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am having some issues with HAProxy configuration. I have been playing around with it to try and make it more resilient to high server loads and Denial of Service. However, I felt it was working fine until suddenly I was victim of a (D)DoS attack - Haproxy was reporting the backend as down even though I could still access it fine via the direct port.

Could somebody please check my HAProxy config and see if there is somewhere I am messing up or why I would be experiencing this.. I just can't seem to understand why this is happening.

Thanks in advance (and after of course) .

global

    # Global Max Connections
        maxconn                                 20000
        # Various Other Settings
        pidfile                                 /var/run/haproxy.pid
        stats socket                            /var/run/haproxy.stat mode 600 level admin
        stats timeout                           5m
        chroot                                  /usr/share/haproxy
        daemon
        # User Settings
        user                                    haproxy
        group                                   haproxy

defaults
        # Default configuration settings for Haproxy
        retries                                 2
        maxconn                                 19500
        timeout server                          10s
        timeout client                          10s
        timeout queue                           10s
        timeout connect                         10s
        timeout http-request                    10s
        # Error files
        errorfile 503 /etc/phpconf/haErrors/503.http

frontend Connection_Handler
        default_backend Primary
        bind :80
        mode                                    http
        option                                  forwardfor
        option                                  http-server-close
        maxconn                                 20000
        # Check if cookie exists
        #acl cookie_set hdr_sub(cookie) authorized=1
        # If cookie doesn't exist try and set it
        #redirect prefix * set-cookie authorized=1 if !cookie_set

        # If the cookie is still not set, send it to blocked backend
        #use_backend Cookie_Block if !cookie_set

        ## (D)DoS Mitigation ##
        # Setup stick table
        stick-table type ip size 1m expire 10m store gpc0
        # Configure the DoS src
        acl src_DoS src_get_gpc0(Connection_Handler) gt 0
        # Use DoS tarpit if src_DoS
        use_backend DoS_Tarpit if src_DoS
        # If not blocked, track the connection
        tcp-request connection track-sc1 src if ! src_DoS

listen Statistics_Engine
        mode http
        bind                                    XX.XXX.XX.XX:9012
        stats                                   enable
        stats uri                               /admin?stats=true

        stats auth                              admin:Password
        stats hide-version
        stats refresh 2s
        #stats scope # Add this option to provide stats for a singular backend

backend Primary
        # Option Configs
        option                                  httpclose
        option                                  redispatch
        option                                  abortonclose

        ## (D)DoS Mitigation ##
        # The following table is recording the IP, connection rate and bytes out rate
        stick-table type ip size 200k expire 10s store conn_rate(5s)

        # Track request and enforce rules
        tcp-request content track-sc2 src
        # Mark as abuse if exceeding connection rate
        acl conn_rate_abuse sc2_conn_rate gt 80
        # Mark as abuse if over X bytes
        acl data_rate_abuse sc2_bytes_out_rate gt 200000

        # Set ACL rule to enforce on frontend
        acl mark_as_DoS sc1_inc_gpc0 gt 0
        # Block connections marked as DoS
        tcp-request content reject if conn_rate_abuse mark_as_DoS
        #tcp-request content reject if data_rate_abuse mark_as_DoS

        # Configure Server
        mode http
        option forwardfor
        server Primary_HTTP 0.0.0.0:1080 check addr 127.0.0.1 port 80 inter 3000 rise 2 fall 3 maxconn 20000
        #fullconn 1024

backend Conn_Tarpit
        # Tarpit for connections
        mode http
        timeout tarpit 20s
        reqitarpit .
        errorfile 503 /etc/phpconf/haErrors/tarpit_503.txt

backend Cookie_Block
        # Block connections that will not take on a cookie
        mode http
        reqdeny .
        errorfile 503 /etc/phpconf/haErrors/503_cookie.txt

backend DoS_Tarpit

        # Tarpit for suspected attacks
        log 127.0.0.1 local1 info
        timeout tarpit 10s # Tarpit for 10 seconds
        errorfile 500 /etc/phpconf/haErrors/500_DoS.txt
        mode http
        reqitarpit .
share|improve this question

1 Answer 1

I see nothing obviously wrong in your config, you seem to have already taken care of correctly tuning your settings (especially maxconn). Is conntrack loaded on this machine ? The connection table might be full, preventing checks and connections from establishing to the server.

Also, have you checked how many concurrent connections were sent to the server ? It's possible that the server is alternatively going up and down due to the load.

Check the kernel log messages for any unexpected error.

share|improve this answer
2  
Hi Willy. Thanks for that, I appreciate the reply. Due to regular denial of service I have had to modify conntrack and the table from memory is just about 10k. The attack was interesting as it was not exactly large but was requesting a large file primarily to consume bandwidth I guess. Current connections and load were all fine, I was monitoring the attack as it went and realised haproxy was causing problems. I stopped haproxy and put Litespeed back on port 80 and the attack was handled fine - That is what I cant get my head around. Seems haproxy is falsely reporting it as down. –  Chris Jan 30 '11 at 8:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.