Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have downloaded and installed RabbitMQ in cluster on Windows:

  • downloaded and installed ERLANG and RABBITMQ on two servers (for example Alpha and Bravo)
  • installed management plugin, set custom tcp_listener port (to increase security) and add custom admin login instead of default guest:guest login
  • set same cookie on both machines (in both folders C:\Windows\ and C:\Users\username)
  • installed RabbitMQ as Windows Service
  • joined Bravo node to Alpha node to cluster
  • added policy to mirror both all queues (Pattern="", Definition="ha-mode: all")
  • I created durable mirrored queue and publish message to it - it is synchronized between both nodes (it works fine as I expected)

But then I tried to test what will happen if I turn off one node:

  • I turned off node Bravo
  • I added message to Alpha node
  • I turned back on node Bravo
  • missing message was synchronized to Bravo node but in management console this node was marked as "unsynchronized"

What is wrong this queue? But there are other problems. For example:

  • I turned off node Bravo
  • I added message to Alpha node
  • I turned back on node Bravo and waited to synchronize message
  • I turned off Alpha node
  • I added message to Bravo node
  • I turned Alpha node back on
  • all added messages in queue magically disappeared

This is really wierd. Anybody has idea what happened?

Also there is third test case:

  • I have turned off Alpha node, added message to Bravo and turn also Bravo node off
  • then I turned on Alpha node and I recieved message "timeout_waiting_for_tables". I tried it more times. It started to work only when I turned also Alpha node back on

Maybe I just didn't get how clustering works in RabbitMQ. Can anyone help me and tell what is going on?

share|improve this question

1 Answer 1

up vote 0 down vote accepted

Same problem but there is a little understanding to be done as well as a possible pitfall.

Firstly, I was fooled by the fact that I didn't pass my vhost to the command:

rabbitmqctl set_policy -p myvhost HA '*' '{"ha-mode": "all"}'

Otherwise the vhost defaults to "/"

After this, when I logged onto the web console, I saw that the node field was reporting on two nodes ...now. Great :-)

However, if you bring one up and down, then the other up and down, the queue disappears!? This is because there is NO "synchronisation" in the mirroring, ONLY "stacking". Meaning if you bring a node down, the rest of the messages are served from the remaining node (or nodes). If you bring a new/existing node up, it will only mirror NEW messages that are added.

I'm fairly new to this so I would assume that having 3 nodes would be far better than two. This means that if one node goes down, there is still resiliance over the other two nodes (depending what your biz case is right). Of course if two nodes go down, you have lost replication for anything left in the queues. I reckon this should be called the "3 strike setup"!

share|improve this answer
    
Thank you, Richie –  Tomáš Jecha Mar 21 '13 at 11:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.