Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
0
votes
1 answer

How to correctly use BlockingCollection.GetConsumingEnumerable?

I'm trying to implement a producer/consumer pattern using BlockingCollection so I've written up a simple console application to test it. public class Program { public static void Main(string[] args) { var workQueue = new…
user9993
  • 5,833
  • 11
  • 56
  • 117
0
votes
1 answer

start simple 1-producer 2-consumers through executor service

public class SemActionPlace { public SemMonitor StartConsumerProducer() { SemMonitor monitor = new SemMonitor(); List threads = new LinkedList<>(); Thread p1 = new Thread(new Producer(monitor), "P1"); …
user7787220
0
votes
1 answer

Unity multithreading producer consumer

I created a situation where a producer class adds strings to a queue in a controller class. The controller class notifies the worker class that there is work and executes the work, in this case appending strings to a file. I'm not able to get this…
0
votes
1 answer

How to implement this specific Producer-Consumer pattern

I'm trying to write a windows service whose producers and consumers work like this: Producer: At scheduled times, get all unprocessed items (Processed = 0 on their row in the db) and add each one to the work queue that isn't already in the work…
user7127000
  • 3,143
  • 6
  • 24
  • 41
0
votes
1 answer

Is pipe or BufferedReader in java likely to loose data?

Aim To read all the logs from apache server and store on s3 Background We have following statement in the httpd.conf ErrorLog "| /usr/bin/tee -a /var/log/httpd/error_log | /usr/bin/java -cp…
Albatross
  • 669
  • 7
  • 24
0
votes
2 answers

A strange bug about a Java concurrency program (using semaphore)

I ran into the problem that every time the program will stop executing when the producer fill all the places in the arraylist buffer. But, theoretically. the consumer process could still get into the get() function, cause the notEmpty semaphore got…
0
votes
1 answer

Monitor in my java program gets into a deadlock

I'm trying to solve single consumer/producer problem using monitor in Java, and the code is as follows. When I run this code, it will finally get stucked. The most typical case is that the consumer calls wait(), and then the producer keeps producing…
Harold H.
  • 57
  • 5
0
votes
1 answer

Why can I never empty the BlockingCollection?

I am downloading some JSON periodically, say every 10 seconds... When the data arrives, an event is fired. The event fired simply adds the JSON to a BlockingCollection(to be processed). I'm trying to process the JSON as fast as possible (as…
pookie
  • 3,796
  • 6
  • 49
  • 105
0
votes
3 answers

Producer consumer problems without semaphores in java threads synchronization

Hi I have been trying to solve the producer consumer problem in java without semaphores. When I use single producer and single consumer then my code is working fine. But when I add more than one consumer then it is completely messing up, all the…
0
votes
1 answer

producer-consumer between two children forked [C]

I've a problem with this famous issue. I must fork two child (producer and consumer) that communicate thanks to a pipe. The first child(producer) must read strings from stdin, send them to the second child(consumer) that must convert them to…
Zanarkand
  • 3
  • 8
0
votes
1 answer

What is the most efficient way of creating a unique list of incoming documents through Kafka when compared with those in ElasticSearch?

In ElasticSearch, I will have an index type of RSS documents, each with their own hash. Next, I have a scheduler that retrieves a list of RSS documents from a feed through Kafka Connect, to add as a microservices broker. Using the BulkRequestBuilder…
ElHaix
  • 12,846
  • 27
  • 115
  • 203
0
votes
1 answer

Any benefits of using Looper for Producer-Consumer pattern?

In my app there are few threads with while-on-volatile-boolean loops and BlockingQueues for quite simple objects-messages they process (pipelined architecture). Recently I've encountered android.os.Looper class with it's friends, that seems to do…
Pavlus
  • 1,651
  • 1
  • 13
  • 24
0
votes
0 answers

How to achieve a more complicated Producer-Consumer Scenario

First, Assuming a scene like this: { Queue1: [taskA1, taskA2, taskA3, ... ] } { Queue2: [taskB1, taskB2, ... ] } ... { QueueN: [taskN1, ... ] } These queues contains varies number of tasks, where tasks are dynamically pushed. Secondly, we…
0
votes
1 answer

C++ Producer-Consumer with exception handling to prevent deadlock

In classical producer-consumer problem, we have producers wait when the shared buffer is full and consumers wait when the shared buffer is empty. We have two posix threads, one producer and other worker, synchroinizing with one another using…
Dr. Debasish Jana
  • 6,980
  • 4
  • 30
  • 69
0
votes
1 answer

Why do my consumer threads stop before my producer threads are done?

I have recently written a bounded lock free queue and was making some tests for it. In the test, some threads produce prime numbers (by starting at some number, counting up by 6 times the number of pairs of producer threads, checking every number…
hacatu
  • 638
  • 6
  • 15