Deadlock when waiting from multiple threads on same std::condition_variable_any::wait_for with std::stop_token

Question

In an application I'm writing I have a threading model which is a simplified as the following:

• a generator jthread (m_WorkerGenerator) is starting async tasks.
• the multiple async tasks work until the the generator thread is stopped. For this, they use the reference of the std::stop_token (m_token) and they wait on the same condition_variable_any (m_cv), locked under the same mutex (m_mut). The deadlock happens before the std::jthread::request_stop() is called on the m_WorkerGenerator.

#include <iostream>
#include <condition_variable>
#include <thread>
#include <chrono>
#include <future>

using namespace std::chrono_literals;

class Foo {
std::condition_variable_any m_cv;
std::mutex m_mut;
std::stop_token m_token;
std::jthread m_WorkerGenerator;

void worker() {
    std::cout << "Worker thread start" << std::endl;

    while (true) {
        std::unique_lock lck{ m_mut };
        if (m_cv.wait_for(lck, m_token, 5ms, [=]() { return m_token.stop_requested(); })) {
            break;
        }
    }

    std::cout << "Worker thread stop" << std::endl;
}

public:

Foo() {
    m_WorkerGenerator = std::jthread{ [&](std::stop_token t) {
        m_token = t;

        std::vector<std::future<void>> futures;
        while (!t.stop_requested()) {

            auto fut = std::async(std::launch::async, [=]() {
                worker();
                });
            futures.emplace_back(std::move(fut));

            std::this_thread::sleep_for(5ms);
        }
    } };
}
};

int main()
{
    Foo f;
    std::this_thread::sleep_for(50ms); // Increase here if you can't reproduce
}

If I rewrite using the condition_variable_any::wait_for without the stop_token and signaling it manually from the stop_callback the deadlock is not happening.

#include <iostream>
#include <condition_variable>
#include <thread>
#include <chrono>
#include <future>

using namespace std::chrono_literals;

class Foo {
std::condition_variable_any m_cv;
std::mutex m_mut;
std::stop_token m_token;
std::jthread m_WorkerGenerator;

void worker() {
    std::cout << "Worker thread start" << std::endl;

    while (true) {
        std::unique_lock lck{ m_mut };
        if (m_cv.wait_for(lck, 5ms, [=]() { return m_token.stop_requested(); })) {
            break;
        }
    }

    std::cout << "Worker thread stop" << std::endl;
}

public:

Foo() {
    m_WorkerGenerator = std::jthread{ [&](std::stop_token t) {
        m_token = t;

        std::stop_callback(t, [=]() {
            m_cv.notify_all();
            });

        std::vector<std::future<void>> futures;
        while (!t.stop_requested()) {

            auto fut = std::async(std::launch::async, [=]() {
                worker();
                });
            futures.emplace_back(std::move(fut));

            std::this_thread::sleep_for(5ms);
        }
    } };
}
};

int main()
{
    Foo f;
    std::this_thread::sleep_for(5000ms);
}

Answer 1

This is a typical Lost Wakeup.

In a nutshell and just to self-check, ask yourself each time, what makes you think that any worker will be executed at least once till the wait_for point while you endlessly create new ones in Foo::Foo ? They could and randomly do, but, again, what makes you think they must do so? Nobody guarantees this even with this sleep_for. So, formally this code could overflow memory at some point.

If case the article is not enough and you want more specifics and solution approach, see the:

Why 'wait with predicate' solves the 'lost wakeup' for condition variable?