1

In an ActiveMQ Artemis HA cluster (v2.13.0), the HA policy’s <scale-down/> property causes the backup broker to move all pending messages to one of the remaining live instances if the master instance fails. Once drained, the slave broker stops itself and waits for the master instance to go live again. This behavior effectively reduces the number of broker pairs in a cluster, even if just one master broker fails. Right?

When the <scale-down/> property is not set, the slave broker becomes live on master failure, and the number of live brokers in a cluster does not change. However, I am wondering if a live backup instance still moves pending messages to other brokers if shutdown intentionally (i.e., SIGTERM). This behavior would preserve all messages in case of intentional broker shutdown while keeping the number of live brokers in case of failures.

Improve this question

1 Answer 1

Reset to default
2

This behavior effectively reduces the number of broker pairs in a cluster, even if just one master broker fails. Right?

That's correct.

When the property is not set, the slave broker becomes live on master failure, and the number of live brokers in a cluster does not change.

That's also correct.

However, I am wondering if a live backup instance still moves pending messages to other brokers if shutdown intentionally (i.e., SIGTERM).

If scale-down is not configured then no messages will be moved.

The scale-down functionality was really designed for cloud use-cases where live-only clusters would grow and shrink based on demand. When the cluster shrinks the broker that's going away needs a way to preserve its messages so it dumps them to another broker in the cluster. Furthermore, cloud infrastructure typically makes HA configuration moot due to the management and redundancy provided by the cloud itself. For example, if a node fails then the cloud infrastructure can simply restart it. In this case the broker can simply re-attach to whatever storage it was using before and all the durable messages which were there before the crash (or whatever) will still be there.

In my opinion, while it is technically possible to use scale-down in an HA configuration with a live and a backup I don't think it makes much sense to do so since the live and backup already share message data (either via shared-storage or replication).

Finally, it's worth noting that scale-down can be performed administratively using the ActiveMQServerControl MBean.

Improve this answer
3
  • Thanks for your explanation, much appreciated. What I’m trying to address is really both use-cases, scaling and high availability. While it is true that for cloud environments the general availability is high, I have seen numerous node failures in different setups (e.g., kernel panics, admin errors, JVM kills due to out-of-memory, etc.) that I would like to address for a use-case where message loss is not acceptable. Is there a manual way to initiate message draining which can be triggered as a pre-stop hook?
    – Stephan
    Jun 25, 2020 at 7:17
  • I don't see why any of the conditions you listed would necessarily lead to message loss with or without HA. Typically in a cloud environment if a node fails then the cloud infrastructure will simply restart it. In this case the broker can simply re-attach to whatever storage it was using before and all the durable messages which were there before the crash (or whatever) will still be there.
    – jbertram
    Jun 28, 2020 at 15:53
  • I updated my answer to provide additional clarity. Hope that helps!
    – jbertram
    Jun 28, 2020 at 16:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .