0

I am trying to implement message ordering in Apache Artemis cluster. Producers/Consumers connecting to the cluster implement High availability. So at one point of time there will be two instances of same application connecting to topic or queue. So far, I could find following two methods which can be used to achieve ordering in Red Hat AMQ / Artemis cluster:

  1. Message Groups (is reliable only when there is one consumer per node in the cluster as per the documentation)
  2. Exclusive Queues (Message order is preserved only on a single node).

I fully understand using cluster and expecting message ordering are conflicting requirements, but its still a requirement to be implemented in the project I am working on as the consumers are unable to handle the complexity of handling out of order messages.

What are the alternatives to the above that can be used to implement message ordering in Artemis ActiveMQ / Red Hat AMQ Cluster?

Tim Bish
  • 17,475
  • 4
  • 32
  • 42
akki
  • 423
  • 2
  • 12
  • Can you elaborate as to why clustering & ordered messaging is a requirement? – Justin Bertram Feb 15 '21 at 15:47
  • We are using enterprise wide cluster which is used by polygot teams. Even though we can use only 1 master/slave pair in the cluster it is still not highly available since its not spawning across data centers. In short we have no control on the topology. Some applications are able to handle out of order messages on the consumer side while other cannot. Also the consumer applications have High Availability deployment with Active/Active configuration, hence multiple consumers start consuming disturbing the order of messages at the source. – akki Feb 17 '21 at 07:16

1 Answers1

1

In a clustered environment an exclusive consumer is tied to each queue on each broker, so depending on the message load balancing configuration, you could have one consumer per broker receiving messages. This breaks ordering. Your producer and consumers need to be on the same broker instance to preserve ordering.

As you say, clustered message groups functionality is not reliable, nevertheless it may be useful in some, very specific use cases. Also note that, for HA it's not sufficient to have a cluster of brokers. You need to explicitly configure HA (shared-store or replication), otherwise, if a broker fails permanently, then all its messages would be lost.

fvaleri
  • 638
  • 1
  • 4
  • 10
  • I tried this example as per the documentation, If we have an exclusive queue configured in a cluster of 2 nodes. One consumer C1 connects to this exclusive queue on node 1 and another consumer C2 connects to the same exclusive queue on node 2. Lets assume 2 messages are sent by the producer. The first message reaches C1 (and this consumer takes long to acknowledge). In the mean time consumer C2 receives the second message, so the end result is the order of messages is not guaranteed. How can we tackle this? – akki Feb 15 '21 at 08:27
  • As I said, do not use *clustered* groups and configure your clients to always connect to the same cluster broker. That way, you can use *local* groups which works reliably and with better performance. – fvaleri Feb 15 '21 at 08:54
  • This example I mentioned above is about exclusive queues and not clustered groups correct me if wrong? – akki Feb 15 '21 at 08:57
  • 1
    Updated my answer adding your test use case. – fvaleri Feb 15 '21 at 10:56