2

What I expect for the queue is following:

Let say, the queue contains A-B-C-D-E. And I have 3 consumers. What I need is ALL consumers consume the item A before dequeue A and move on to consume B.

That's to say, all the consumers get ALL the items in the queue. I have the trade-off solution, which is use 3 ArrayBlockingQueue for 3 consumers. But I'd know to know whether there's such queue.

Maroun
  • 94,125
  • 30
  • 188
  • 241
macemers
  • 2,194
  • 6
  • 38
  • 55

6 Answers6

3

Requirements you describe is publish/subscribe design pattern. It is implemented by JMS providers such as ActiveMQ.

Evgeniy Dorofeev
  • 133,369
  • 30
  • 199
  • 275
1

No, there isn't. Sounds like a pretty easy thing to implement, though. Some options that jump to mind are:

  1. Include a CountDownLatch in the object to consume, and discard the item only when it reaches 0
  2. Have two of the consumers not consume the item (peek the queue rather than pop it)
  3. Don't use consumers, use listeners

Or if you're willing to use something outside of rt.jar, you could use a messaging queue. ActiveMQ and RabbitMQ are popular, they both support publish-subscribe.

EDIT: brief description of listener pattern

Instead of multiplie consumers, have a single ListenerManager consumer. It pulls an object from the queue, then passes it to all Listeners that have previously registered themselves with the ListenerManager. Finally, the ListenerManager disposes of the object. Have a single ListenerManager per queue, and a single queue per event/object type. This makes it easy to manage.

Paul Hicks
  • 13,289
  • 5
  • 51
  • 78
0

You can use disruptor with multiple consumers. Please follow the link. You might have to specify the dependency between consumers. Please use this example code I tried to attach Two handlers with one queue

Rohit Sachan
  • 1,178
  • 1
  • 8
  • 16
  • I'm not familiar with Disruptor. Would you explain more about how disruptor solve this problem? – macemers Feb 05 '14 at 07:24
  • @user838204 You can find much of the documents about disruptor here http://stackoverflow.com/questions/6933347/how-should-one-use-disruptor-disruptor-pattern-to-build-real-world-message-sys OR http://mechanitis.blogspot.sg/2011/08/dissecting-disruptor-why-its-so-fast.html – Rohit Sachan Feb 05 '14 at 08:19
  • Thx very much. After looking your codes, the Disruptor is clearly a solution to my problem with good performance. – macemers Feb 07 '14 at 02:44
0

Is there any reason why the simple solution of just putting n references to item A into the queue instead of just one wouldn't work? If you have multiple producers you would have to synchronize their access, but if it's important that a finishes before b you need that anyhow already.

That's actually at least as memory efficient as using an additional class to handle this - the instance of the class adds at least 2 words overhead, while the proposed one only adds 2 references (which can be smaller than 2 words).

Voo
  • 29,040
  • 11
  • 82
  • 156
  • I don't understand of "putting n references to item A into the queue"? Please explain a little bit. And I have only one produce for this case. – macemers Feb 06 '14 at 05:47
0

What you need to do is to have a single thread listen on the queue, which will then spread the item to the consumers.

// listener thread
Element element;
while ((element = queue.take()) != null) {
  for (final Listener listener : myListeners) {
    listener.getQueue().put(element);
  }
}

Basically a Producer-pattern combined with an Observer pattern.

TwoThe
  • 13,879
  • 6
  • 30
  • 54
  • It's the same as what Paul Hicks describes – macemers Feb 07 '14 at 02:46
  • In point 3, yes, but he suggest the use of external libraries, so I felt like pointing out that this can easily be done with pure Java in about 3 lines of code as well. – TwoThe Feb 07 '14 at 08:34
0

Use a separate queue for every consumer, and let the producer add it to all of them. There's not really a drawback here.

It's not clear if you need all consumers to have consumed A before any of them moves to B, in that case you have to adjust.

Philip Frank
  • 225
  • 3
  • 5