Questions tagged [producer-consumer]

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue.

The Producer-Consumer Problem (also known as the bounded-buffer problem) is a classical example of a multi-process synchronization problem. The problem describes two processes, the producer and the consumer, who share a common, fixed-size buffer used as a queue. The producer's job is to generate a piece of data, put it into the buffer and start again. At the same time the consumer is consuming the data (i.e., removing it from the buffer) one piece at a time. The problem is to make sure that the producer won't try to add data into the buffer if it's full and that the consumer won't try to remove data from an empty buffer.

The solution for the producer is to either go to sleep or discard data if the buffer is full. The next time the consumer removes an item from the buffer, it notifies the producer who starts to fill the buffer again. In the same way, the consumer can go to sleep if it finds the buffer to be empty. The next time the producer puts data into the buffer, it wakes up the sleeping consumer. The solution can be reached by means of inter-process communication, typically using semaphores. An inadequate solution could result in a deadlock where both processes are waiting to be awakened. The problem can also be generalized to have multiple producers and consumers.

References

1564 questions
0
votes
1 answer

Observing changes in a BlockingCollection without consuming

A consumer thread and multiple producer threads synchronize their work with a System.Collections.Concurrent.BlockingCollection. The producers call blockingCollection.Add() and the consumer runs foreach (var item in…
HugoRune
  • 13,157
  • 7
  • 69
  • 144
0
votes
3 answers

thread safe producer/consumer pattern in c++

How Can I develop a producer/ consumer pattern which is thread safe? in my case, the producer runs in a thread and the consumer runs on another thread. Is std::deque is safe for this purpose? can I push_back to the back of a deque in one thread and…
mans
  • 17,104
  • 45
  • 172
  • 321
0
votes
0 answers

In ruby, how could I trigger a program when redis is not empy?

I use redis list as a queue, i also have a consumer program take the stuff from redis. I want to have the consumer in ruby take stuff from redis as long as redis is not empty. Do you know how could i achieve this? Right now, I manually run the…
xin buxu1
  • 397
  • 4
  • 16
0
votes
1 answer

rabbitmq(pika) throws an exception when use RPC

RabbiMQ RPC throws an exception when i build publisher-consumer pattern. This is my code: sender.py # coding:utf-8 import pika import uuid class MessageSender(object): def __init__(self): self.connention = pika.BlockingConnection( …
Hugo
  • 33
  • 6
0
votes
1 answer

For producer-consumer pattern which implementation is better : ArrayBlockingQueue or Arraylists with Exchangers

Usecase : its a producer consumer pattern where order (FIFO) is to be maintained. As of now there is single producer and a single consumer to process data in sequential manner to maintain order, but in future ( due to removal of some business…
Dhaval D
  • 1,087
  • 2
  • 14
  • 25
0
votes
0 answers

BlockingCollection is taking too much time to wake up

I have an issue when BlockingCollection is taking too much time to wake up from .Take() call. The scenario is this: I have a thread that pushing data to BlockingCollection very fast (actually in XUnit i did a for loop. I have a 3 Task that…
Alex F
  • 3,180
  • 2
  • 28
  • 40
0
votes
2 answers

One thread does not run till the other one is ended

I am making a simple producer consumer problem. The waiter is the producer and the chef is the consumer. I have implemented it such that the Chef thread waits till Waiter notifies it of an incoming order in the Queue. public class Main { public…
0
votes
2 answers

Producer / Consumer - I/O Disk

I have a compressed file in the disk, that a partitioned in blocks. I read a block from disk decompress it to memory and the read the data. It is possible to create a producer/consumer, one thread that recovers compacted blocks from disk and put in…
p.magalhaes
  • 7,595
  • 10
  • 53
  • 108
0
votes
1 answer

file writing using blockingcollection

I have a tcp listener which listens and writes data from the server. I used a BlockingCollection to store data. Here I don't know when the file ends. So, my filestream is always open. Part of my code is: private static BlockingCollection
tony montana
  • 105
  • 1
  • 13
0
votes
1 answer

What's the most efficient way to handle infinite tasks in Producer/Consumer?

I have Gigabytes of data (stored in messages, each message is about 500KB) in a cloud queue (Azure) and data keeps coming. I need to do some processing on each message. I've decided to create 2 background workers, one to get the data into memory,…
Shmoopy
  • 5,334
  • 4
  • 36
  • 72
0
votes
1 answer

Implementing the procducer-consumer pattern with .NET 4.0

With alle the new paralell programming features in .NET 4.0, what would be a a simple and fast way to implement the producer-consumer pattern (where at least one thread is producing/enqueuing task items and another thread executes/dequeues these…
bitbonk
  • 48,890
  • 37
  • 186
  • 278
0
votes
2 answers

Referencing a Producer Thread from a Consumer Thread

I'm currently developing a programme using multiple producer threads, and one consumer thread. I am wondering if there is an easy way to reference exactly which Producer Thread the Consumer Thread has consumed from. Here's an example of my output…
Gavin Friel
  • 191
  • 1
  • 3
  • 13
0
votes
1 answer

Understand the producer and receiver using coroutine in python

I want to use coroutine to implement producer and receiver. My idea is using two coroutines , one for producer and one for recevier. But my understand for coroutine's send and running mode is wrong. Here is my code : def coroutine(func): def…
jiamo
  • 1,406
  • 1
  • 17
  • 29
0
votes
1 answer

Deleting data in atomic / fenced data

Assume we use the standard consumer/producer pattern in our C++11 program: (from: http://en.cppreference.com/w/cpp/atomic/memory_order) #include #include #include #include std::atomic ptr; int…
atlaste
  • 30,418
  • 3
  • 57
  • 87
0
votes
1 answer

pthread API equivalents of semaphores in semaphore.h

This question is related to a solution to the producer/consumer problem I'm working on. We were asked to use pthreads to solve the problem. In order to create my empty and full semaphores for the bounded buffer, I used semaphore.h, but I was told…
Victor Brunell
  • 5,668
  • 10
  • 30
  • 46