2

Setup

I am setting up a HA cluster for a web application with 2 nodes (2 physical servers):

  • node1 (current master node)
  • node2 (current slave node)

Using Corosync & Pacemaker I was able to create the cluster and some resources agents including an IP failover and a Webserver (apache).

Resources

  • Failover resource exists on only one node at a time

Uses a python script to make API calls to my hosting provider in order to update the IP failover destination

  • WebServer resource exists (as a clone) on every available node

Standard OCF resource using Apache's server-status handler

Constraints

  • There is a constraint that says that Failover and WebServer must be running at the same time on a server in order for considering it as available.

The issue

Now I would like to create a custom resource agent (like I did for the IP failover) that will:

  • Switch the mysql instance of the current slave node into a master node
  • Switch the mysql instance of the current master node into a slave node of the new master node
  • Basically do the same for Redis instance

Ideally, the resource would be started on only one node (master), and stopped on all other nodes (slaves). Therefore, starting the resource would put the current node in master mode, and stopping it would put it in slave mode.

I made a script that can easily achieve all of these operations. Here's how it works.

Turn local node in master mode:

# /usr/local/bin/db_failover_switch.sh master

Turn local node in slave mode:

# /usr/local/bin/db_failover_switch.sh slave 123.45.67.89

The synopsis is pretty straightforward to understand. The problem I am facing, is that I obviously need to set the master IP in order for the slave to configure MySQL and Redis services accordingly.

TL;DR

In case of failover, I want:

  • Resource starts on node2 which becomes master node
  • Resource stops on node1 which becomes slave node

In order to stop the resource (i.e. setting it into slave mode), I need to know the IP address (hostname will do) of the node which has the resource running.

Is there a way I can have a dynamic parameter that Pacemaker will pass to my resource agent? Or maybe can I retrieve the clusters information directly from my resource agent script to know which is the node running a specific resource?

  • Which cluster shell are you using? crmsh? cman? – gf_ Feb 2 '16 at 17:15
  • I'm using crm. But now that I had the time to look deeper into it, I'm starting to think I'm doing this the wrong way. I should probably set up 2 different multiple-state resources for MySQL and Redis, and use heartbeat mysql and redis resources agents. – Jordan Becker Feb 2 '16 at 17:22
1

After reading your comment I'm not sure if you still want to go that route you had planned for originally, but anyway, here is an input for this:

Using crm_mon --group-by-node -1 you could get a "one-time" view of the current state of your cluster, grouped by your nodes. The -1 parameter makes this non-interactive, which means it just displays the data and then exits.

(Edit: Maybe using crm_mon -1 makes parsing in your specific case more easy.)

You could parse this output and act accordingly.

(Personal note: I would go as well the route you've described in your comment. After all, these are two daemons, so two resources seems reasonable. Also, the resource agents to use exist already. Good luck!)

  • I already explored that idea though, but awkwardly, crm_mon (be it interactive or not), does not give more information on nodes other than the node name in the cluster. So yes, I could find out which node is running as master, and use some hard-coded table to establish an associative array... But that's way too heavy. I'll go with the separated resources agents that already exist in heartbeat, like you pointed out. By the way, interesting to see how they retrieve master's IP – Jordan Becker Feb 2 '16 at 17:51
  • @JordanBecker In my setups, I'm using the hostname as the name of the node, but yeah, it seems a bit "hackish". – gf_ Feb 2 '16 at 17:58
  • That would for sure be better than my current "primary" and "secondary" node names... But for now it's the least of my concerns, I'm having a hard time trying to setup properly the MySQL master/slave setup... Will probably ask another question on SF... – Jordan Becker Feb 2 '16 at 19:16
  • @JordanBecker Ping me if you do, maybe I could help. – gf_ Feb 2 '16 at 19:33
  • I just opened a new question over here, would be great if you could have a look! – Jordan Becker Feb 2 '16 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.