Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
0
votes
0 answers

Java: Producer - consumer scenario with non-uniform consumers

I have currently a tough time with implementing the following scenario in Java: I have a single producer with multiple consumer problem - but not not every consumer can process every job: There are two types of jobs: xxl-jobs require a very large…
JMax
  • 1,134
  • 1
  • 11
  • 20
0
votes
1 answer

How to pass message to consumer without broker

Producer Different Thread with its own SynchronizedBlockingQueue. Each producer put there message into its own queue. Consumer Different Thread which will get message from either from any one of queue and start process. Now for communicating…
0
votes
1 answer

ActiveMQ (Java) - Checking the amount of time a connection has been active (i.e. Consumer or Producer)

I am currently using Apache ActiveMQ as a JMS solution to send various messages back and forth between clients. I would like to check the uptime for specific clients (i.e. Consumer or Producers), but I have not found a way to do this so far. I…
J Dack.
  • 23
  • 4
0
votes
0 answers

Apache Kafka config/producer.properties

I want to copy kafka data to hadoop. How can i do configuration of producer.properties file. Kafka data format is .avro file. metadata.broker.list request.required.acks producer.type serializer.class
0
votes
4 answers

producer-consumer using synchronization

I wrote code to implement the Producer-Consumer problem and it seems to be working fine without the need to synchronize.Is it possible? How do I test the code and check if it is actually working right or not? How do I know if deadlock will occur?…
collegian
  • 65
  • 1
  • 7
0
votes
1 answer

How to implement semaphores with threads without global variables

My current code answers the producer consumer question, but I want to implement it in a different way. I do not want to use a global variable to keep track of the count in my buffer stack . What should my thought process be? typedef int…
Damenace
  • 1
  • 2
0
votes
1 answer

Reuse Boost thread (from threadpool) after interruption or join

At the moment I am using a producer consumer model for the rendering portion of a realtime graphics application. The consumer will continually look for data in our queue(infinite loop); however I am fearful that this may cause my simulation to get…
triple
  • 421
  • 4
  • 14
0
votes
0 answers

Sharing memory buffer in a producer consumer semaphore

I have a question on how to implement using a shared memory buffer in semaphore sets. I am doing the classic consumer producer problem where I add a new item to a buffer stack and depending on the scheduler I want to remove the first in item…
Damenace
  • 1
  • 2
0
votes
0 answers

C++ - Multiple producer multiple consumer, producing for a specific consumer without busywaiting

So I'm working on a job system where threads in a pool pull jobs off of a queue and run them, and each job can potentially queue other jobs. The queue is currently lockless, and the threads all wait on a semaphore that gets incremented as jobs are…
0
votes
1 answer

Kafka is slow to produce messages in first seconds

i m working with kafka, and i made a producer like that: synchronized (obj) { while (true){ long start = Instant.now().toEpochMilli(); for (int i=0; i< NUM_MSG_SEC ; i++) { …
FrankTan
  • 1,626
  • 6
  • 28
  • 63
0
votes
1 answer

Producer Consumer scenario, halt reading from queue till execution is complete

I am implementing a producer consumer scenario using a BlockingQueue. This is my consumers run method: @Override public void run() { try { Message msg; System.out.println("Consumer started"); …
User3
  • 2,465
  • 8
  • 41
  • 84
0
votes
1 answer

pthread_cond_wait() not waiting, returning int 6

Classic producer consumer program. Currently trying to have the consumer wait if the queue is empty. The queue is a struct in memory. I'm having an issue with my pthread_cond_wait() function. We are not allowed global variables so I'm storing the…
john stamos
  • 1,054
  • 5
  • 17
  • 36
0
votes
1 answer

boost::async_write large files and memory consumption

I'm writing an Http Server using boost::asio. For large files, in order to avoid reading the whole file into memory and sending it to the network, I read it part by part which I send on the network using boost::asio::async_write. The problem is that…
Ben D
  • 465
  • 1
  • 6
  • 20
0
votes
1 answer

How to use the same fifo for more consumers and producers?

I created the two producers and two consumers, producer1 sent to consumer1 two integer number and consumer1 print the sum of numbers, and I have another consumer and producer, producer2 sent to consumer2 the path for folder and consumer2 print the…
Laurentiu
  • 39
  • 6
0
votes
0 answers

Producer and consumer - segmentation fault

I'm a beginner in concurrent programming and I want to implement a consumer and producer. I want to send after 3 seconds from producer two integers and I want to print the sum of numbers from producer. After running the code I have Segmentation…
Laurentiu
  • 39
  • 6