Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
0
votes
0 answers

BlockingQueue skips/loses elements

Context: As I understand, BlockingQueue is used to pass data between threads aka producer-consumer problem. For example, in my case I use it to store received DatagramPackets so that other thread can process them. For now it just logs all incoming…
0
votes
1 answer

No endpoint could be found for: test, please check your classpath contains the needed Camel component jar

I am trying to send and receive messages using akka-camel and created a sample example for producer and consumer like below : Producer: import akka.actor.{Actor, ActorSystem, Props} import akka.camel.Producer class CamelJmsProducer extends Actor…
Jet
  • 3,018
  • 4
  • 33
  • 48
0
votes
1 answer

Producer consumer semaphore value is not correct

I'm trying to implement producer-consumer problem in C using processes and System V IPC and I'm stuck on one thing. This is early version of my code (without implementing queue operations or even producer and consumer executing in loop) that I was…
beti
  • 13
  • 1
  • 3
0
votes
1 answer

Wait for exactly n works done: Producer, consumer and counter

The problem i face is i have a thread A and n works need to be done. Thread A must wait until this n works are completely done. My idea is using a CountDownLatch with n count and use Producer/Consumer pattern to control the Worker. I use an…
robinmag
  • 17,520
  • 19
  • 54
  • 55
0
votes
1 answer

Producer-Consumer, Bounded Buffer Insert Error?

I am attempting to do an assignment that requires having multiple producers and consumers producing and consuming random integers in a bounded buffer. Here is my code: import java.util.Deque; import java.util.LinkedList; import…
0
votes
0 answers

Producer/Consumer using Boost.Interprocess (shared memory) using condition_variable

I do have to dive into concurrency with C++11 at the moment and I am having a very hard time to get a basic producer/consumer to work with Boost.Interprocess using shared memory. I've already tried a lot of different things, but I can't make two…
0
votes
1 answer

Delta loading of data in Apache Kafka

I have this scenario where I need to fetch millions of records from an Oracle database and then need to send these records in a chunk of 1000 to an Apache Kafka producer. While fetching the records the next time, I have to avoid pulling the already…
0
votes
1 answer

multiple operators sharing 3 tools, how to avoid deadlock?

I would like to ask some suggestions to solve this particular deadlock: 3 operators, each is given 2 of 3 different materials at random: 1, 2, 3 3 tools, each material needs a different tool to process by an operator. material 1 needs tool 1 …
Di Wang
  • 471
  • 1
  • 8
  • 22
0
votes
0 answers

Semaphores - Consumer Producer weird result

I recently learned Semaphore and Consumer Producer problem in class and encountered this question: I used two semaphores to indicate the empty and the full in a buffer plus a mutex. #include #include #include…
Joji
  • 4,703
  • 7
  • 41
  • 86
0
votes
3 answers

BlockingQueue: put() and isEmpty() do not work together?

I would like to have a SynchronousQueue where I insert elements from one thread with put(), so the input is blocked until the element is taken in another thread. In the other thread I perform lots of calculations and from time to time want to check…
martinus
  • 17,736
  • 15
  • 72
  • 92
0
votes
1 answer

How to make changes to or call a method in a running thread in Python?

I have a producer thread that produces data from a serial connection and puts them into multiple queues that will be used by different consumer threads. However, I'd like to be able to add in additional queues (additional consumers) from the main…
0
votes
1 answer

.NET/C# - What is the best option instead of an ActionBlock (or Channel) for speed?

corefxlab has something called a Channel which is a really nice implementation of an async P-C queue and definitely does what I'm looking for. I'm curious if there's an implementation that ultimately had a similar API to ActionBlock: Must be…
Oren Ferrari
  • 89
  • 1
  • 10
0
votes
0 answers

How does the structure of the producer/consumer process work?

I'm currently studying how producer/consumer process works. Here we are given the following shared data structures, mutex, empty, and full. We have the codes for the producer, consumer, wait, and signal. Below those code is my assumption. I try to…
asilvester635
  • 81
  • 2
  • 13
0
votes
0 answers

Blocking queue - Is client side locking needed?

As mentioned by Java_author: 5.1.1. Problems with Synchronized Collections The synchronized collections are thread-safe, but you may sometimes need to use additional client-side locking to guard compound actions. Example - Multiple…
overexchange
  • 15,768
  • 30
  • 152
  • 347
0
votes
1 answer

One Producer ten consumers file-processing with Executors.newSingleThreadExecutor()

I have a LinkedBlockingQueue with an arbitrarily picked capacity of 10, and an input file with 1000 lines. I have one ExecutorService-type variable in the main method of the service class that, to my knowledge, first handles--using…