Is there a "single slot" queue?

Question

I need to use a queue which holds only one element, any new element discarding the existing one. Is there a built-in solution?

The solution I coded works but I strive not to reinvent the wheel :)

import Queue

def myput(q, what):
    # empty the queue
    while not q.empty():
        q.get()
    q.put(what)

q = Queue.Queue()
print("queue size: {}".format(q.qsize()))
myput(q, "hello")
myput(q, "hello")
myput(q, "hello")
print("queue size: {}".format(q.qsize()))

EDIT: following some comments & answers -- I know that a variable is just for that :) In my program, though, queues will be used to communicate between processes.

try [deque][1] [1]: https://docs.python.org/2/library/collections.html#collections.deque — sundar nataraj, Jun 30 '14 at 10:53
@alko: could be :) but I will be using several processes and the queue will be used to communicate — WoJ, Jun 30 '14 at 10:55
What if between calling `q.empty()` and `q.get()` your consumer also calls `q.get()`? You will deadlock if `q.get()` is a blocking operation. — Steven Rumbalski, Jun 30 '14 at 13:06
@StevenRumbalski: yes, please see [Will's answer](http://stackoverflow.com/a/24489511/903011) below. — WoJ, Jun 30 '14 at 13:11

score 3 · Accepted Answer · answered Jun 30 '14 at 11:51

3

As you specify you are using queues to communicate between processes, you should use the multiprocesssing.Queue.

In order to ensure there is only one item in the queue at once, you can have the producers sharing a lock and, whilst locked, first get_nowait from the queue before put. This is similar to the loop you have in your code, but without the race condition of two producers both emptying the queue before putting their new item, and therefore ending up with two items in the queue.

answered Jun 30 '14 at 11:51

Will

73,905
40
169
246

You are right. In my previous comment I mentioned that I would have only two processes, one adding and the other reading from the queue but it is not true, as both will be reading (the first one - to empty the queue). I will make sure to implement the lock. Thanks. – WoJ Jun 30 '14 at 13:10
@WoJ multiprocessing.Queue is safe; you only need a lock (to enforce your max-length=1 constraint) if you have multiple producers. Multiple consumers do not need a lock. – Will Jun 30 '14 at 13:31
4

Or attack the problem from the other end and have consumer `get` until empty and then just work the last successful `get`. – Steven Rumbalski Jun 30 '14 at 13:31
@StevenRumbalski ... which is an excellent idea :) I will do that. I will then have strictly one producer and one consumer. – WoJ Jun 30 '14 at 13:34

Daniel · Answer 2 · 2018-10-12T08:04:20.090

Although the OP is regarding inter-process-communication, I came across a situation where I needed a queue with a single element (such that old elements are discarded when a new element is appended) set up between two threads (producer/consumer).

The following code illustrates the solution I came up with using a collections.deque as was mentioned in the comments:

import collections
import _thread
import time

def main():

    def producer(q):
       i = 0
       while True:
           q.append(i)
           i+=1
           time.sleep(0.75)


    def consumer(q):
        while True:
            try:
                v = q.popleft()
                print(v)
            except IndexError:
                print("nothing to pop...queue is empty")
                sleep(1)

    deq = collections.deque(maxlen=1)
    print("starting")
    _thread.start_new_thread(producer, (deq,))
    _thread.start_new_thread(consumer, (deq,))


if __name__ == "__main__":
    main()

In the code above, since the producer is faster than the consumer (sleeps less), some of the elements will not be processed.

Notes (from the documentation):

Deques support thread-safe, memory efficient appends and pops from either side of the deque with approximately the same O(1) performance in either direction.

Once a bounded length deque is full, when new items are added, a corresponding number of items are discarded from the opposite end.

Warning: The code never stops :)

Is there a "single slot" queue?

2 Answers2