0

I'm using activemq (pretty new to me) and I have one producer and several consumers.

Thing is, I want to address messages for specific consumers. I read about selectors but also read that it is a bad practice to use and also read about some alternatives.

The alternative sounds good for me, but I'm not sure how I can create these queues for each and every one of my slaves. Each of my slaves has an ID (uuid) that I can use when I'm creating the listener - like this:

<jms:listener-container
        container-type="default"
        connection-factory="jmsConnectionFactory"
        acknowledge="auto">
    <jms:listener destination="slave.tasks.${slave.id}" ref="jmsActivityListener" method="onMessage" />
</jms:listener-container>

This requires the slave.properties file to contain the following entry:

slave.id=XXXXXXX

My questions are:

1) Is that the way to do it (defining a queue per consumer)?

2) How can I generate this salve.id value (I don't want the user to fill it as it has to be unique)?

Thanks

Community
  • 1
  • 1
Noam
  • 3,049
  • 10
  • 34
  • 52

1 Answers1

0

Virtual Topics to the rescue

The idea behind virtual topics is that producers send to a topic in the usual JMS way. Consumers can continue to use the Topic semantics in the JMS specification. However if the topic is virtual, consumer can consume from a physical queue for a logical topic subscription, allowing many consumers to be running on many machines & threads to load balance the load. E.g., let's say we have a topic called VirtualTopic.Orders. (Where the prefix VirtualTopic. indicates its a virtual topic). And we logically want to send orders to systems A and B. Now with regular durable topics we'd create a JMS consumer for clientID_A and "A" along with clientID_B and "B". With virtual topics we can just go right ahead and consume to queue Consumer.A.VirtualTopic.Orders to be a consumer for system A or consume to Consumer.B.VirtualTopic.Orders to be a consumer for system B. We can now have a pool of consumers for each system which then compete for messages for systems A or B such that all the messages for system A are processed exactly once and similarly for system B.

http://activemq.apache.org/virtual-destinations.html

<jms:listener-container
        container-type="default"
        connection-factory="jmsConnectionFactory"
        acknowledge="auto">
    <jms:listener destination="Consumer.#{T(java.lang.System).currentTimeMillis()}.VirtualTopic.tasks" ref="jmsActivityListener" method="onMessage" />
</jms:listener-container>

your producer can send messages to topic "VirtualTopic.tasks" and these messages will be sent to all Consumer.*.VirtualTopic.tasks queues

Hassen Bennour
  • 3,885
  • 2
  • 12
  • 20
  • Thanks for the reply, but I think this will not solve my problem - since I would like to send a message to a specific consumer (not all of them). If I understand this correctly, this is more of a Topic solution which is something I want to avoid, since the message has to be sent to one consumer (out of, say, 100) so I wouldn't want to use a topic (among other reasons) – Noam Feb 14 '17 at 13:58
  • if you use a user/pwd in jmsConnectionFactory with admins role your destinations will be auto created at runtime, but you need to find a solution to map each consumer to their id – Hassen Bennour Feb 14 '17 at 14:16
  • Yes - I know that and I am using the admins creds. but I need a way to configure the listeners in the slave side - for that matter, I need to know the id which is not part of the configuration (as I mentioned in my question) – Noam Feb 14 '17 at 14:19
  • You can use advisory http://activemq.apache.org/advisory-message.html or jmx to list queues created or connected consumers but what kind of information let you decide to which consumer you want to send the message ?? – Hassen Bennour Feb 14 '17 at 17:10
  • My slaves send information to my Master, when they do - they append their ids to these objects. When the Master wishes to execute a task on one of these objects - it has to know to which slave to address that request (that data exist on these objects) – Noam Feb 15 '17 at 09:18
  • so if the master knows the id of the slave for who he want to send a message, if a consumer creates a random id the problem is if he stops and restarts the id will change and he will lost the messages on the last queue created for him, i think that the id need to be static and fixed for each consumer if you want your consumer reconnects to the same queue after restart, crash... but if not you can use my proposition to create a random id and add it to all messages as a replyTo header property – Hassen Bennour Feb 15 '17 at 10:00
  • another solution is to create a consumers manager to distribute the id's and maps them to consumers and give the same id if a consumer restarts but this is depends in your architecuture. – Hassen Bennour Feb 15 '17 at 10:05
  • The consumer's id is persisted and it doesn't change so no problem there. The only problem that I have now, is that I don't know how to register a listener in a dynamic manner - currently I'm checking if this can help: http://stackoverflow.com/questions/32332840/spring-dynamically-create-jmstemplates – Noam Feb 15 '17 at 10:43
  • yep your link contains the right code to create a DefaultMessageListenerContainer's dynamically – Hassen Bennour Feb 15 '17 at 11:00