1

I have 2 virtual servers on Softlayer, both running HAProxy. I am trying to set up failover with keepalived. Each server has a private IP and public IP, and they are on the same VLAN. I've tried many different settings for keepalived, but stopping HAProxy on master, it does not failover to the BACKUP.

I read that multicast was not supported, so I have since changed my settings to unicast. The settings now, on backup/master are basically this:

vrrp_script chk_haproxy {
  script "pidof haproxy"
  interval 2
}

vrrp_instance VI_1 {
  debug 2
  interface eth1
  state MASTER
  virtual_router_id 51
  priority 101
  unicast_src_ip 1.2.3.4        # My IP
  unicast_peer {
    5.6.7.8                   # peer IP
  }
  track_script {
    chk_haproxy
  }
}

Where MYIP is the public IP address of the server the conf file is on, and PEERIP is the public IP address of the peer. Still it is not working. Stopping HAProxy on the master, it does not failover to the backup.

I am wondering if anyone has set up HAProxy with failover on Softlayer, and how they went about accomplishing it?

5

I managed to get this set up, and here's how I did it:

  1. I created a global IP address using SoftLayer's control panel.

  2. I have Debian 7 on both HAProxy virtual servers. I added the global IP address to eth1 interface on both servers.

  3. Here's the HAProxy settings used on both servers:

    global
      log 127.0.0.1 local0
      log 127.0.0.1 local1 notice
      maxconn 4096
      user haproxy
      group haproxy
    
    defaults
      log global  
      mode http    
      option httplog 
      option dontlognull
      retries 3
      maxconn 2000
      option redispatch
      timeout connect 5000
      timeout client 50000
      timeout server 50000
      stats uri / haproxy
    
    listen webfarm 0.0.0.0:80
      mode http
      stats enable
      stats uri /haproxy?stats
      stats realm Haproxy\ Statistics
      stats auth haproxy:stats
      balance roundrobin
      cookie LBN insert indirect nocache
      option httpclose
      option forwardfor
      server app1-west <public_ip>:8080 cookie node1 check
      server app2-west <public_ip>:8080 cookie node2 check 
    
  4. Here's the Keepalived settings on MASTER server:

    global_defs {
        notification_email {
            admin@mydomain.com
        }
        notification_email_from me@me.com
        smtp_server 127.0.0.1
        smtp_connect_timeout 30
        router_id LB_MASTER_ACTIVE
    }
    
    # Define the script used to check if haproxy is still working
    vrrp_script chk_haproxy {
        script "killall -0 haproxy"   # verify the pid existance
        interval 2                    # check every 2 seconds
        weight 2                      # add 2 points of prio if OK
    }
    
    # Virtual interface.
    vrrp_instance VI_1 {
        state MASTER
        interface eth1
        virtual_router_id 51
        priority 101
        smtp_alert 
    
        authentication {
            auth_type PASS
            auth_pass 1111 #replace with random string
        }
    
        vrrp_unicast_bind <my_private_ip>
        vrrp_unicast_peer <peers_private_ip>
    
        # Check if HAProxy is running or not.
        track_script {
            chk_haproxy
        }
        notify_master /usr/bin/reroute_global
    }
    
  5. Here's the Keepalived settings on BACKUP server:

    global_defs {
        notification_email {
            admin@mydomain.com
        }
        notification_email_from me@me.com
        smtp_server 127.0.0.1
        smtp_connect_timeout 30
        router_id LB_BACKUP_PASSIVE
    }
    
    # Define the script used to check if haproxy is still working
    vrrp_script chk_haproxy {
        script "killall -0 haproxy"   # verify the pid existance
        interval 2                    # check every 2 seconds
        weight 2                      # add 2 points of prio if OK
    }
    
    # Virtual interface.
    vrrp_instance VI_1 {
        state BACKUP
        interface eth1
        virtual_router_id 51
        priority 100
        smtp_alert 
        advert_int 1
    
        authentication {
            auth_type PASS
            auth_pass 1111 #replace with random string
        }
    
        vrrp_unicast_bind <my_private_ip>
        vrrp_unicast_peer <peers_private_ip>
    
        # Check if HAProxy is running or not.
        track_script {
            chk_haproxy
        }
        notify_master /usr/bin/reroute_global
    }
    
  6. As said above, I am running Debian 7. As can be seen in keepalived settings, I have a notify_master script. Here's everything needed to get the script running:

    apt-get install cpanminus libssl-dev build-essential libxml2-dev libexpat1-dev
    cpanm SOAP::Lite XML::Hash::LX IO::Interface
    git clone https://github.com/softlayer/softlayer-api-perl-client.git
    mv softlayer-api-perl-client/SoftLayer /usr/share/perl5
    
  7. Now that all the dependencies are in place, the script should work. Here's the script, which I saved as /usr/bin/reroute_global:

    #!/usr/bin/env perl
    use strict;
    use warnings;
    
    use SoftLayer::API::SOAP;
    use IO::Interface::Simple;
    
    # SoftLayer API Information
    my $api_user = 'YOUR_API_USERNAME';
    my $api_key  = 'YOUR_API_KEY';
    
    # Get the IP address associated with eth1
    my $if   = IO::Interface::Simple->new('eth1');
    
    # Create client object to SoftLayer_Account
    my $client = SoftLayer::API::SOAP->new('SoftLayer_Account', undef, $api_user, $api_key);
    
    # Get global IP address ID of first global IP address.
    my $global_ip_id = $client->getGlobalIpRecords()->result->[0]->{id};
    
    # Create client object to SoftLayer_Network_Subnet_IpAddress_Global
    $client = SoftLayer::API::SOAP->new('SoftLayer_Network_Subnet_IpAddress_Global', $global_ip_id, $api_user, $api_key);
    
    # Reroute global IP address to this systems public IP
    $client->route($if->address);
    

You'd need to change API_USERNAME/KEY to match your API credentials. The script grabs the first global IP from your SoftLayer global IP addresses, and then reroutes the global IP to the system. In the case of a failover, the BACKUP becomes MASTER and runs the script, which routes the global IP address to itself.

Testing

  1. curl http://<global_IP>
  2. On master server, service haproxy stop
  3. On backup: tail -f /var/log/syslog. You should see something like this:

    Feb 12 01:11:55 proxy2-west Keepalived_vrrp[11816]: VRRP_Script(chk_haproxy) succeeded
    Feb 12 01:11:55 proxy2-west Keepalived_vrrp[11816]: SMTP alert successfully sent.
    Feb 12 01:12:29 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) forcing a new MASTER election
    Feb 12 01:12:29 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) forcing a new MASTER election
    Feb 12 01:12:30 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) Transition to MASTER STATE
    Feb 12 01:12:31 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) Entering MASTER STATE
    Feb 12 01:12:31 proxy2-west Keepalived_vrrp[11816]: Opening script file /usr/bin/reroute_global
    
  4. curl http://<global_IP> (it should work if failover worked)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.