boost::lockfree::spsc_queue busy wait strategy. Is there a blocking pop?

Question

So i'm using a boost::lockfree::spec_queue to communicate via two boost_threads running functors of two objects in my application.

All is fine except for the fact that the spec_queue::pop() method is non blocking. It returns True or False even if there is nothing in the queue. However my queue always seems to return True (problem #1). I think this is because i preallocate the queue.

typedef boost::lockfree::spsc_queue<q_pl, boost::lockfree::capacity<100000> > spsc_queue;

This means that to use the queue efficiently i have to busy wait constantly popping the queue using 100% cpu. Id rather not sleep for arbitrary amounts of time. I've used other queues in java which block until an object is made available. Can this be done with std:: or boost:: data structures?

Answer 1

A lock free queue, by definition, does not have blocking operations.

How would you synchronize on the datastructure? There is no internal lock, for obvious reasons, because that would mean all clients need to synchronize on it, making it your grandfathers locking concurrent queue.

So indeed, you will have to devise a waiting function yourself. How you do this depends on your use case, which is probably why the library doesn't supply one (disclaimer: I haven't checked and I don't claim to know the full documentation).

So what can you do:

• As you already described, you can spin in a tight loop. Obviously, you'll do this if you know that your wait condition (queue non-empty) is always going to be satisfied very quickly.

• Alternatively, poll the queue at a certain frequency (doing micro-sleeps in the mean time). Scheduling a good good frequency is an art: for some applications 100ms is optimal, for others, a potential 100ms wait would destroy throughput. So, vary and measure your performance indicators (don't forget about power consumption if your application is going to be deployed on many cores in a datacenter :)).

Lastly, you could arrive at a hybrid solution, spinning for a fixed number of iterations, and resorting to (increasing) interval polling if nothing arrives. This would nicely support servers applications where high loads occur in bursts.

Answer 2

Use a semaphore to cause the producers to sleep when the queue is full, and another semaphore to cause the consumers to sleep when the queue is empty. when the queue is neither full nor empty, the sem_post and sem_wait operations are nonblocking (in newer kernels)

#include <semaphore.h>

template<typename lock_free_container>
class blocking_lock_free
{
public:
    lock_free_queue_semaphore(size_t n) : container(n)
    {
        sem_init(&pop_semaphore, 0, 0);
        sem_init(&push_semaphore, 0, n);
    }

    ~lock_free_queue_semaphore()
    {
        sem_destroy(&pop_semaphore);
        sem_destroy(&push_semaphore);
    }

    bool push(const lock_free_container::value_type& v)
    {
        sem_wait(&push_semaphore);
        bool ret = container::bounded_push(v);
        ASSERT(ret);
        if (ret)
            sem_post(&pop_semaphore);
        else
            sem_post(&push_semaphore); // shouldn't happen
        return ret;
    }

    bool pop(lock_free_container::value_type& v)
    {
        sem_wait(&pop_semaphore);
        bool ret = container::pop(v);
        ASSERT(ret);
        if (ret)
            sem_post(&push_semaphore);
        else
            sem_post(&pop_semaphore); // shouldn't happen
        return ret;
    }
private:
    lock_free_container container;
    sem_t pop_semaphore;
    sem_t push_semaphore;
};