1

I have 2 virtual servers on Softlayer, both running HAProxy. I am trying to set up failover with keepalived. Each server has a private IP and public IP, and they are on the same VLAN. I've tried many different settings for keepalived, but stopping HAProxy on master, it does not failover to the BACKUP.

I read that multicast was not supported, so I have since changed my settings to unicast. The settings now, on backup/master are basically this:

vrrp_script chk_haproxy {
  script "pidof haproxy"
  interval 2
}

vrrp_instance VI_1 {
  debug 2
  interface eth1
  state MASTER
  virtual_router_id 51
  priority 101
  unicast_src_ip 1.2.3.4        # My IP
  unicast_peer {
    5.6.7.8                   # peer IP
  }
  track_script {
    chk_haproxy
  }
}

Where MYIP is the public IP address of the server the conf file is on, and PEERIP is the public IP address of the peer. Still it is not working. Stopping HAProxy on the master, it does not failover to the backup.

I am wondering if anyone has set up HAProxy with failover on Softlayer, and how they went about accomplishing it?

Improve this question

1 Answer 1

Reset to default
5

I managed to get this set up, and here's how I did it:

  1. I created a global IP address using SoftLayer's control panel.

  2. I have Debian 7 on both HAProxy virtual servers. I added the global IP address to eth1 interface on both servers.

  3. Here's the HAProxy settings used on both servers:

    global
  log 127.0.0.1 local0
  log 127.0.0.1 local1 notice
  maxconn 4096
  user haproxy
  group haproxy

defaults
  log global  
  mode http    
  option httplog 
  option dontlognull
  retries 3
  maxconn 2000
  option redispatch
  timeout connect 5000
  timeout client 50000
  timeout server 50000
  stats uri / haproxy

listen webfarm 0.0.0.0:80
  mode http
  stats enable
  stats uri /haproxy?stats
  stats realm Haproxy\ Statistics
  stats auth haproxy:stats
  balance roundrobin
  cookie LBN insert indirect nocache
  option httpclose
  option forwardfor
  server app1-west <public_ip>:8080 cookie node1 check
  server app2-west <public_ip>:8080 cookie node2 check

  4. Here's the Keepalived settings on MASTER server:

    global_defs {
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id LB_MASTER_ACTIVE
}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "killall -0 haproxy"   # verify the pid existance
    interval 2                    # check every 2 seconds
    weight 2                      # add 2 points of prio if OK
}

# Virtual interface.
vrrp_instance VI_1 {
    state MASTER
    interface eth1
    virtual_router_id 51
    priority 101
    smtp_alert 

    authentication {
        auth_type PASS
        auth_pass 1111 #replace with random string
    }

    vrrp_unicast_bind <my_private_ip>
    vrrp_unicast_peer <peers_private_ip>

    # Check if HAProxy is running or not.
    track_script {
        chk_haproxy
    }
    notify_master /usr/bin/reroute_global
}

  5. Here's the Keepalived settings on BACKUP server:

    global_defs {
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id LB_BACKUP_PASSIVE
}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "killall -0 haproxy"   # verify the pid existance
    interval 2                    # check every 2 seconds
    weight 2                      # add 2 points of prio if OK
}

# Virtual interface.
vrrp_instance VI_1 {
    state BACKUP
    interface eth1
    virtual_router_id 51
    priority 100
    smtp_alert 
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass 1111 #replace with random string
    }

    vrrp_unicast_bind <my_private_ip>
    vrrp_unicast_peer <peers_private_ip>

    # Check if HAProxy is running or not.
    track_script {
        chk_haproxy
    }
    notify_master /usr/bin/reroute_global
}

  6. As said above, I am running Debian 7. As can be seen in keepalived settings, I have a notify_master script. Here's everything needed to get the script running:

    apt-get install cpanminus libssl-dev build-essential libxml2-dev libexpat1-dev
cpanm SOAP::Lite XML::Hash::LX IO::Interface
git clone https://github.com/softlayer/softlayer-api-perl-client.git
mv softlayer-api-perl-client/SoftLayer /usr/share/perl5

  7. Now that all the dependencies are in place, the script should work. Here's the script, which I saved as /usr/bin/reroute_global:

    #!/usr/bin/env perl
use strict;
use warnings;

use SoftLayer::API::SOAP;
use IO::Interface::Simple;

# SoftLayer API Information
my $api_user = 'YOUR_API_USERNAME';
my $api_key  = 'YOUR_API_KEY';

# Get the IP address associated with eth1
my $if   = IO::Interface::Simple->new('eth1');

# Create client object to SoftLayer_Account
my $client = SoftLayer::API::SOAP->new('SoftLayer_Account', undef, $api_user, $api_key);

# Get global IP address ID of first global IP address.
my $global_ip_id = $client->getGlobalIpRecords()->result->[0]->{id};

# Create client object to SoftLayer_Network_Subnet_IpAddress_Global
$client = SoftLayer::API::SOAP->new('SoftLayer_Network_Subnet_IpAddress_Global', $global_ip_id, $api_user, $api_key);

# Reroute global IP address to this systems public IP
$client->route($if->address);

You'd need to change API_USERNAME/KEY to match your API credentials. The script grabs the first global IP from your SoftLayer global IP addresses, and then reroutes the global IP to the system. In the case of a failover, the BACKUP becomes MASTER and runs the script, which routes the global IP address to itself.

Testing

  1. curl http://<global_IP>
  2. On master server, service haproxy stop

  3. On backup: tail -f /var/log/syslog. You should see something like this:

    Feb 12 01:11:55 proxy2-west Keepalived_vrrp[11816]: VRRP_Script(chk_haproxy) succeeded
Feb 12 01:11:55 proxy2-west Keepalived_vrrp[11816]: SMTP alert successfully sent.
Feb 12 01:12:29 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) forcing a new MASTER election
Feb 12 01:12:29 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) forcing a new MASTER election
Feb 12 01:12:30 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) Transition to MASTER STATE
Feb 12 01:12:31 proxy2-west Keepalived_vrrp[11816]: VRRP_Instance(VI_1) Entering MASTER STATE
Feb 12 01:12:31 proxy2-west Keepalived_vrrp[11816]: Opening script file /usr/bin/reroute_global
  4. curl http://<global_IP> (it should work if failover worked)
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .