C++ threads: cannot unlock mutex in array after condition_variable wait

Question

I am trying to synchronize one main thread with N children threads. After some reading, I used condition_variable and unique_lock. However, I always get the errors condition_variable::wait: mutex not locked: Operation not permitted or unique_lock::unlock: not locked: Operation not permitted, in OS X. In Linux, I get Operation not permitted only.

To be clearer: my goal is to get a sequence of prints:

main thread, passing to 0
thread 0, passing back to main
main thread, passing to 0
thread 0, passing back to main
...

for each of the four threads.

I adapted the code from the example in http://en.cppreference.com/w/cpp/thread/condition_variable. This example uses unlock after wait, and it works wonderfully with only one thread other than main (N=1). But when adapted to work with N>1 threads, the error above happens.

Yam Marcovic said in the comments that I should not use unlock. But then, why does the cppreference example use it? And why does it work well with one main and one other threads?

Here is the code:

#include <cstdio>
#include <thread>
#include <mutex>
#include <condition_variable>

using namespace std;

constexpr int N_THREADS = 4;
constexpr int N_ITER = 10;

bool in_main[N_THREADS] = {false};

void fun(mutex *const mtx, condition_variable *const cv, int tid){
    for(int i=0; i<N_ITER; i++) {
        unique_lock<mutex> lk(*mtx);
        // Wait until in_main[tid] is false
        cv->wait(lk, [=]{return !in_main[tid];});
        // After the wait we own the lock on mtx, which is in lk
        printf("thread %d, passing back to main\n", tid);
        in_main[tid] = true;
        lk.unlock(); // error here, but example uses unlock
        cv->notify_one();
    }
}

int main(int argc, char *argv[]) {
    // We are going to create N_THREADS threads. Create mutexes and
    // condition_variables for all of them.
    mutex mtx[N_THREADS];
    condition_variable cv[N_THREADS];
    thread t[N_THREADS];
    // Create N_THREADS unique_locks for using the condition_variable with each
    // thread
    unique_lock<mutex> lk[N_THREADS];
    for(int i=0; i<N_THREADS; i++) {
        lk[i] = unique_lock<mutex>(mtx[i]);
        // Create the new thread, giving it its thread id, the mutex and the
        // condition_variable,
        t[i] = thread(fun, &mtx[i], &cv[i], i);
    }

    for(int i=0; i < N_ITER*N_THREADS; i++) {
        int tid=i % N_THREADS; // Thread id
        // Wait until in_main[tid] is true
        cv[tid].wait(lk[tid], [=]{return in_main[tid];});
        // After the wait we own the lock on mtx[tid], which is in lk[tid]
        printf("main thread, passing to %d\n", tid);
        in_main[tid] = false;
        lk[tid].unlock(); // error here, but example uses unlock
        cv[tid].notify_one();
    }
    for(int i=0; i<N_THREADS; i++)
        t[i].join();
    return 0;
}

Sample output:

thread 0, passing back to main
main thread, passing to 0
thread 1, passing back to main
thread 0, passing back to main
main thread, passing to 1
thread 2, passing back to main
thread 1, passing back to main
main thread, passing to 2
thread 2, passing back to main
thread 3, passing back to main
main thread, passing to 3
main thread, passing to 0
thread 3, passing back to main
libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: unique_lock::unlock: not locked: Operation not permitted
Abort trap: 6

Answer 1

you are trying to unlock your mutexes many times! look at the code carefully:

 for(int i=0; i < N_ITER*N_THREADS; i++) {
        int tid=i % N_THREADS; // Thread id

where N_ITER is 10 and N_THREADS is 4 always, because they are constexpr
we get:

 for(int i=0; i < 40; i++) {
        int tid=i % 4; // Thread id

so, when i = 0 the mutex in lk[0] is unlocked, and then when i=4 then tid = 4%4 so again tid = 0 and you are unlocking it again! std::system_error is thrown in this case.

plus, why are all of these C-Pointers anyway? it's not like anyof them can be null at any time.. switch to references..

also, usually when dealing with array indexes the convention is to use size_t and not int.

Answer 2

I found what the problem is. This question Using std::mutex, std::condition_variable and std::unique_lock helped me.

Constructing a unique_lock is acquiring the unique_lock too. So it must be done inside the loop, just before calling wait. The function fun looks the same, but main now looks like this:

int main(int argc, char *argv[]) {
    // We are going to create N_THREADS threads. Create mutexes and
    // condition_variables for all of them.
    mutex mtx[N_THREADS];
    condition_variable cv[N_THREADS];
    thread t[N_THREADS];
    // Create N_THREADS unique_locks for using the condition_variable with each
    // thread
    for(int i=0; i<N_THREADS; i++) {
        // Create the new thread, giving it its thread id, the mutex and the
        // condition_variable,
        t[i] = thread(fun, &mtx[i], &cv[i], i);
        // DO NOT construct, therefore acquire, a unique_lock
    }

    for(int i=0; i < N_ITER*N_THREADS; i++) {
        int tid=i % N_THREADS; // Thread id
        // Acquire the unique_lock here
        unique_lock<mutex> lk(mtx[tid]);
        // Wait until in_main[tid] is true
        cv[tid].wait(lk, [=]{return in_main[tid];});
        // After the wait we own the lock on mtx[tid], which is in lk[tid]
        printf("main thread, passing to %d\n", tid);
        in_main[tid] = false;
        lk.unlock(); // error here, but example uses unlock
        cv[tid].notify_one();
    }
    for(int i=0; i<N_THREADS; i++)
        t[i].join();
    return 0;
}

The only difference is that the unique_lock is constructed inside the loop.