Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
6
votes
2 answers

Java: Thread producer consumer what is the most efficient way to wait for data to be produced

When using BlockingQueue to consume data that is produced what is the most efficient method for waiting for the data to appear? Scenario: Step 1) The data list will be a data store where timestamps are added to. These timestamps are required to be…
perkss
  • 1,037
  • 1
  • 11
  • 37
6
votes
3 answers

Suspend consumer in producer/consumer pattern

I have producer and consumer connected with BlockingQueue. Consumer wait records from queue and process it: Record r = mQueue.take(); process(r); I need pause this process for a while from other thread. How to implement it? Now I think implement it…
HotIceCream
  • 2,271
  • 2
  • 19
  • 27
6
votes
2 answers

What is the correct way to implement Producer Consumer in scala

I try to implement a Producer Consumer program in scala without employing Queue. Because I think Actor has already implemented "mail queue" or something else, it would be redundant to write the code again. I tried to write the program in Actor…
worldterminator
  • 2,968
  • 6
  • 33
  • 52
6
votes
1 answer

Why are only BUFFER_SIZE-1 items allowed in buffer in Producer-Consumer paradigm using shared memory?

Here is an excerpt from the book "Operating System Concepts" 7th Edition Galvin,Gagne Chapter 3 from the hard-copy itself : The following variables reside in a region of memory shared by the producer and consumer processes: #define BUFFER_SIZE…
6
votes
2 answers

Producer/consumer pattern with a fixed-size FIFO queue

I need to implement the producer/consumer pattern around a fixed-size FIFO queue. I think a wrapper class around a ConcurrentQueue might work for this but I'm not completely sure (and I've never worked with a ConcurrentQueue before). The twist in…
bmt22033
  • 6,880
  • 14
  • 69
  • 98
5
votes
4 answers

Elegant ways to notify consumer when producer is done?

I'm implementing a concurrent_blocking_queue with minimal functions: //a thin wrapper over std::queue template class concurrent_blocking_queue { std::queue m_internal_queue; //... public: void add(T const & item); …
Nawaz
  • 353,942
  • 115
  • 666
  • 851
5
votes
5 answers

consumer/producer in c++

This is a classic c/p problem where some threads produce data while other read the data. Both the producer and consumers are sharing a const sized buffer. If the buffer is empty then the consumers have to wait and if it is full then the producer has…
Dan P.
  • 179
  • 1
  • 2
  • 12
5
votes
4 answers

Producer/consumer of a web crawler using queue with unknown size

I need to crawl parent web pages and its children web pages and I followed the producer/consumer concept from http://www.albahari.com/threading/part4.aspx#%5FWait%5Fand%5FPulse. Also, I used 5 threads which enqueue and dequeue links. Any…
5
votes
4 answers

Create different Objects with same base type. Factory pattern?

I have to implement a multiple producers / multiple consumers example application for a university course and have a hard time to find a solution for the following problem, that doesn't make me feel, that I do something wrong ;) I have to implement…
maruchinu
  • 53
  • 2
  • 5
5
votes
4 answers

Fast C++ single producer single consumer implementation

I'm looking for a single-producer, single-consumer FIFO implementation that would perform faster than the normal lock-write-unlock-signal / waitForSignal-lock-read-unlock stuff. I'm looking for something supported by most POSIX operating systems…
Not Provided
  • 53
  • 1
  • 3
5
votes
2 answers

Choosing a data structure for a variant of producer consumer problem

Right now, I have a queue, with multiple producers and single consumer. Consumer thread operation is slow. Also, consumer takes element from queue through a peek operation, and until the consumption operation is complete, the element cannot be…
Abhijeet Kashnia
  • 12,290
  • 8
  • 38
  • 50
5
votes
3 answers

Algorithm for .NET consumer thread processing of two queues (based on priority)

I have a C# 4.0 app with "high priority" and "low priority" queues implemented as such: BlockingCollection highPriority = new BlockingCollection(1000); BlockingCollection lowPriority = new BlockingCollection(1000); Any data produced in highPriority…
bmt22033
  • 6,880
  • 14
  • 69
  • 98
5
votes
1 answer

ActiveMQ consumer level timeout

I trying to create a consumer level timeout in Active MQ (version 5.15.0). Consider one message is picked by a consumer but not able to acknowledge so in that case i want consumer to timeout so that same message can be picked by other consumer…
subro
  • 104
  • 2
  • 15
5
votes
2 answers

Is TPL Dataflow BufferBlock thread safe?

I have a fairly simple producer-consumer pattern where (simplified) I have two producers who produce output that is to be consumed by one consumer. For this I use System.Threading.Tasks.Dataflow.BufferBlock A BufferBlock object is created. One…
5
votes
2 answers

RabbitMQ consumer overload

I`ve been reading about the principles of AMQP messaging confirms. (https://www.rabbitmq.com/confirms.html). Really helpful and wel written article but one particular thing about consumer aknowledgments is really confusing, here is the…
Sivich
  • 51
  • 1
  • 5