2

We have a set of 30 producers and 30 consumers consuming messages from single ActiveMQ instance currently. All Producers write to 25 queues and all consumers can consume from the queues. There is no message routing or failover in place as of now. We would like to make it horizontally scalable with out any code changes on producers & consumers.

One idea we've had is using two ELB's, one before producers and one before consumers so we would add/remove activemq's at will. I'm unable to find any such documentation or material online for such setup, and would like to know if any one worked on such setup or if its possible in the first place. Any suggestions to make the system horizontally scalable by adding nodes at will will be helpful

zolo
  • 149
  • 2
  • 10

2 Answers2

1

Take a look at networks of brokers. This will let you add activemq instances (presumably using multicast discovery) on the fly. You'll then need to dynamically update the ELB to include the new instances. ActiveMq networks of brokers allow consumers and producers to be connected anywhere on the network and exchange traffic.

Martin Serrano
  • 3,727
  • 1
  • 35
  • 48
  • 1
    There is no multicast in AWS – TheFiddlerWins Nov 17 '15 at 17:06
  • There are alternatives to the multicast discovery. Maybe have the new brokers connect to the (known) addresses of the existing ones with a duplex tcp network connector configuration. The trick will be to make all this dynamic and automatic, if that's what you need. – Duncan Dec 18 '17 at 22:38
1

We went ahead with using automatic broker cluster feature found with ActiveMQ 5.4 or greater. This article has been really helpful in understanding this setup.

zolo
  • 149
  • 2
  • 10
  • Hey zolo, I am interested in setting up the same kind of architecture. So in the end, you don't need ELB for scaling ? The cluster from ActiveMQ can take care of itself ? If not, can you elaborate a bit more. Thank you! – Nathan Do Apr 02 '15 at 04:00
  • The automatic broker cluster seems suitable for high availability, but it you want to scale up (e.g. to support massive numbers of consumers) then I'm not sure the automatic broker cluster is what you need. For that, your original ELB idea seems right. – Duncan Dec 18 '17 at 22:34