Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
0
votes
2 answers

Particular producer-consumer scenario

I need to implement a particular producer-consumer scenario in which the class Consumer creates two thread handling two ports, each one store a value into the corresponding port and, if a value is available on the other port, process is called to…
0
votes
1 answer

couple of question about : wcf self service , consumer/ producer

i have a couple of question. 1. in a wcf self service for inner network process comunication , how many calls can a service handle at the minimum amount of time ? 2. does any one have an open source code for a generic wrapper for…
guyl
  • 2,158
  • 4
  • 32
  • 58
0
votes
2 answers

Processing Huge CSV File using Producer - Consumer Pattern

I am trying to process an arbitary CSV File which can range from 10 records to million records. The CSV File has 4 fixed columns (e.g. a,b,c,d) and 2 additional columns (e,f) which come from external REST API. My goal is to read all the records…
Yogendra
  • 331
  • 5
  • 21
0
votes
0 answers

strtol Implicit conversion

Hello I am having trouble with strtol (specifically, that implicit conversion loses integer precision) as well as with sem_open incompatible pointer types passing sem_t I have tried to use named semaphores instead of unnamed semaphores, use sem_open…
0
votes
1 answer

Make JMS queue aware of the state of the events that are being processed. Is it possible to configure in ActiveMQ?

I'm trying to configure a queue that is aware of the events that are being processed. Questions Does this make sense? :) Is it possible to configure/customize ActiveMQ? Are there any other library that can be "easily" configured to handle such…
vhula
  • 487
  • 2
  • 9
0
votes
1 answer

Condition variable should be used or not to reduce missed wakeups

I have two threads, one is the producer and other is consumer. My consumer is always late (due to some costly function call, simulated in below code using sleeps) so I have used ring buffer as I can afford to loose some events. Questions: I am…
0
votes
1 answer

Pika message consumption is slow while processing messages in callback function

I have a consumer application written in python. It consume the rabbitmq messages through multiprocessing. But when I try to process or validate those messages with in the on_message callback function then the consumption is bit slow. I am opening…
kabik
  • 55
  • 5
0
votes
1 answer

Lockless circular buffer with single producer singular consumer

I have a consumer thread that must never lock nor allocate memory, and a producer thread that can. I want to implement a two place circular buffer to be able to provide data to the consumer thread from the producer, with the bound that whenever no…
0
votes
2 answers

Consumer-driven contract testing with Spring, JMS and ActiveMQ

I am currently trying to learn consumer-driven contract testing in the context of Services that use ActiveMQ Messages to communicate. Spring offers a documentation for Spring Cloud Contract Verifier…
0
votes
1 answer

Producer-consumer scenario causing std::system_error on unique_lock

The following program should create a JobScheduler that puts Jobs into a running_queue from a waiting_queue. When the running_queue is not empty, 2 or more threads execute the Job going to sleep for a time specified into it, if the Job is completed…
Antonio Santoro
  • 827
  • 1
  • 11
  • 29
0
votes
2 answers

Single producer / multiple consumer deadlock

The following code reasults in a deadlock. The problem is that I cannot figure out how unlock the consumers waiting on the condition variable. The consumer should loop and consume from the stack when a certain condition is met. I've tried exiting…
Antonio Santoro
  • 827
  • 1
  • 11
  • 29
0
votes
1 answer

Producer/consumer doesn't generate expected results

I've written such producer/consumer code, which should generate big file filled with random data class Program { static void Main(string[] args) { Random random = new Random(); String filename =…
Sheyko Dmitriy
  • 383
  • 1
  • 3
  • 15
0
votes
1 answer

How to optimize merging elements in parallel

We have following problem. We parsing files (producer) and convert the data into a c# data format. Afterwards we need to merge all of this data together. As this can be done in parallel we startet to implement a producer consumer pattern but…
Lori
  • 155
  • 8
0
votes
1 answer

Looking for an ideal Azure service for fair data processing in Producer - consumer schema

I am trying to manage the situation illustrated on the following picture. We have several clients = producers (also 1 client = 1 to N producers). We need to process the data from each producer fairly (= separately would be the best option), because…
Lkor
  • 434
  • 3
  • 12
0
votes
1 answer

In the scenario of using Wait() and Pulse(), can we replace `while` with `if`?

On the internet, I saw many example about Wait() and Pulse() and they used two while like in this example: class MyQueue { private Queue queue = new Queue(); private const int CAPACITY = 3; public void Put(string…
Phuc
  • 623
  • 1
  • 7
  • 11