1

Summary

I have a single consumer, multiple producer problem where each producer puts its data on a separate queue (because their data types are different), and the producers are all on their own threads.

The consumer, in a loop, will pull items from the producer queues based on their priorities, and then when all the queues are empty, it should go to sleep until more data is produced.

I don't know the best (error-free and fastest) way to put the consumer to sleep and then later to wake it up.

Current Code

Currently I'm using a simple semaphore. Every time new data has been produced the producer calls:

void produce(Data data) {
    queue.add(data);
    semaphore.release();
}

And the consumer:

while (!Thread.currentThread().isInterrupted()) {
    consumeQueues();
    semaphore.tryAcquire(time, TimeUnit.SECONDS);
    semaphore.drainPermits();
}

Current Thoughts

Use an empty selector, have the consumer call selector.select() and the producers call selector.wakeup().

Rewrite all of the produced values and switch to a priority queue. Would be a lot of work and I'm not even sure I could get exactly the same behavior as I have now without losing speed.

Using the lock/conditional approach and calling conditional.await() and conditional.signal(). I'm worried about adding a lot of code and slowing things down with the added lock acquisitions (performance sensitive code).

Preferably I'd just use a static AtomicBoolean that the producers could lazily set to true when data is available, but AtomicBoolean don't have a block-until-true method.

Supun Wijerathne
  • 11,964
  • 10
  • 61
  • 87
billoot
  • 77
  • 15
  • Do you want to wait for all producers to produce data before you move? By the way, Spring has different ways to handle such cases, you may want to try that as well.. – Anand Vaidya Dec 27 '16 at 06:58
  • How you assign priorities for each?? based on the number of items in the queue?? – Supun Wijerathne Dec 27 '16 at 10:30
  • Possible duplicate of [Java - wait on multiple BlockingQueue's in parallel](https://stackoverflow.com/questions/36330623/java-wait-on-multiple-blockingqueues-in-parallel) – Lii Aug 15 '18 at 10:57

1 Answers1

-1

you can use the following structure : a priority queue of blocking queue (or priority queue if there are priority within the same type)

you shuld do an empty interface like ProducedItem that all created items should implement, so it would compile.

PriorityBlockingQueue<PriorityBlockingQueue<ProducedItem>> queue = new PriorityBlockingQueue<PriorityBlockingQueue<ProducedItem>>;

Then when you get a add new produced item:

void addWorkItem(int queueIndex, ProducedItem producedItem) {
    //Note: You may want to make the two operations an atomic operation
    priorityQueues[queueIndex].add(workItem);
    prioritypriorityQueue.add(producers[queueIndex]);
}
Ran Koretzki
  • 464
  • 3
  • 15
  • Thanks for the advice, but like I said before I think priority queues "would be a lot of work and I'm not even sure I could get exactly the same behavior as I have now without losing speed". Priority queues aren't able to re-order themselves if a currently held value changes in priority, it would have to be pulled out and then reinserted. For my purposes that would mean nearly every insertion would make me re-iterate the entire queue when I already have the logic and structures in place to efficiently use my queues. – billoot Dec 23 '16 at 21:17