0

I have a producer consumer model which an arduino generates packets and inside my pc I have a java program which takes those packets and puts them into a BlockingQueue. Now there are threads that process these packets.

Let's say X is producer and A, B and C are consumers thread. there is a queue which all consumers have reference to it. Messages(packets) are immutable objects (i.e. consumers can't change the state of the elements). My question is How can I know all threads are done with specific element inside queue so I can remove it?

here is my consumer run() method:

@Override
public void run()
{
    while (isStarted && !queue.isEmpty()) {
            updateMap(queue.peek());
    }
}

One design I'm thinking of is to use a bounded queue. When producer finds out queue is full then it removes the first element. But I'm not sure if this is a safe approach. I read this tutorial and a few more and what I get is:

Producer should wait if Queue or bucket is full and Consumer should wait if queue or bucket is empty.

I'm sorry if it sounds obvious, but I'm new to multithread programming and the concurrency nature of code sounds scary to me.

EDIT:

All A, B and C do independently. One is for Statistics one for updating network map etc.

EDIT 2:

As @Augusto sugested there is another approach which each of A, B and C has their own queue. My network listener passes the packets to each queue and they process it. It works but How can I do this with only one queue? or is it possible to implement this scenario using only one queue? and if the answer is yes. How and when I need to remove element from queue?

Sevle
  • 3,109
  • 2
  • 19
  • 31
artronics
  • 1,399
  • 2
  • 19
  • 28
  • Do all three of `A`, `B` and `C` need to process the packet in order? If this is the case, you could have 3 queues: Network -> `A` -> `B` -> `C` (where the arrow is a different `ConcurrentLinkedQueue`). If the order doesn't matter, you could put have 3 queues where the network listener puts the Packet (so `A`, `B` and `C` have 'private queues') – Augusto Nov 15 '15 at 15:03
  • @Augusto no, I put the wrong consumer's code. I edited my question. All `A` `B` and `C` acts independently. Is it good to having a separate queue for each threads? – artronics Nov 15 '15 at 15:10
  • If I need a separate queue for each thread, What's the benefit of using `BlockingQueue` at the first place? – artronics Nov 15 '15 at 15:27

1 Answers1

2

(a follow up on my comment after your edit) I would suggest to have a different queue per thread. This is actually a very well know pattern called publish-subscribe.

Taken from the link above:

A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published into the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. Each output channel has only one subscriber, which is only allowed to consume a message once. In this way, each subscriber only gets the message once and consumed copies disappear from their channels.

The main difference between a ConcurrentLinkedQueue and a BlockingQueue is that you can add a limit on how many elements are on the BlockingQueue. This is good in the case that the producer reading from the network generates data faster than the consumers can process. If you use an unbounded queue and this goes for a while you'll end up with an OutOfMemoryError.

Augusto
  • 28,839
  • 5
  • 58
  • 88
  • I think this pattern would be good in an event driven model where different subscribers subscribes on a channel based on their **interests** on **certain events** . In my model however, consumers are **always there** and do **certain things**. I'm not sure if this would be a good idea. seems to me adding unnecessary complexity. – artronics Nov 15 '15 at 15:40
  • beside my original question and in general, in a consumer/producer who is in charge of making more space in a queue? because even by using `unbounded` queue it is not unlimited and at some point we need to remove some elements. – artronics Nov 15 '15 at 15:45
  • So you mean I should use a `ConcurrentLinkedQueue` then each thread does its stuffs and then when the queue reaches to certain size the network watcher must remove the first element. is it correct? – artronics Nov 15 '15 at 15:54
  • My suggestion would be to use a bounded `BlockingQueue` and the elements will be taken out of the queue when you call `take()` (call this instead of `peek()`). – Augusto Nov 15 '15 at 16:22
  • thanks. This will works only if I use separate queue for each thread. The point is when I have different threads accessing **One** queue I can't call `take()` because there might be another thread which needs that element. – artronics Nov 15 '15 at 16:30