pthread_cond_wait not waked up correctly if not joined

Question

The code below was taken from llnl tutorials on pthreads with two modifications:

comment the sleep(1); in function

comment the pthread_join(thread[i],NULL); in function main

/******************************************************************************
* FILE: condvar.c
* DESCRIPTION:
*   Example code for using Pthreads condition variables.  The main thread
*   creates three threads.  Two of those threads increment a "count" variable,
*   while the third thread watches the value of "count".  When "count" 
*   reaches a predefined limit, the waiting thread is signaled by one of the
*   incrementing threads. The waiting thread "awakens" and then modifies
*   count. The program continues until the incrementing threads reach
*   TCOUNT. The main program prints the final value of count.
* SOURCE: Adapted from example code in "Pthreads Programming", B. Nichols
*   et al. O'Reilly and Associates. 
* LAST REVISED: 10/14/10  Blaise Barney
******************************************************************************/
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

#define NUM_THREADS  3
#define TCOUNT 10
#define COUNT_LIMIT 12

int     count = 0;
pthread_mutex_t count_mutex;
pthread_cond_t count_threshold_cv;

void *inc_count(void *t) 
{
  int i;
  long my_id = (long)t;

  for (i=0; i < TCOUNT; i++) {
    pthread_mutex_lock(&count_mutex);
    count++;

    /* 
    Check the value of count and signal waiting thread when condition is
    reached.  Note that this occurs while mutex is locked. 
    */
    if (count == COUNT_LIMIT) {
      printf("inc_count(): thread %ld, count = %d  Threshold reached. ",
             my_id, count);
      pthread_cond_signal(&count_threshold_cv);
      printf("Just sent signal.\n");
      }
    printf("inc_count(): thread %ld, count = %d, unlocking mutex\n", 
       my_id, count);
    pthread_mutex_unlock(&count_mutex);

    /* Do some work so threads can alternate on mutex lock */
    /*sleep(1);*/
    }
  pthread_exit(NULL);
}

void *watch_count(void *t) 
{
  long my_id = (long)t;

  printf("Starting watch_count(): thread %ld\n", my_id);

  /*
  Lock mutex and wait for signal.  Note that the pthread_cond_wait routine
  will automatically and atomically unlock mutex while it waits. 
  Also, note that if COUNT_LIMIT is reached before this routine is run by
  the waiting thread, the loop will be skipped to prevent pthread_cond_wait
  from never returning.
  */
  pthread_mutex_lock(&count_mutex);
  while (count < COUNT_LIMIT) {
    printf("watch_count(): thread %ld Count= %d. Going into wait...\n", my_id,count);
    pthread_cond_wait(&count_threshold_cv, &count_mutex);
    printf("watch_count(): thread %ld Condition signal received. Count= %d\n", my_id,count);
    printf("watch_count(): thread %ld Updating the value of count...\n", my_id,count);
    count += 125;
    printf("watch_count(): thread %ld count now = %d.\n", my_id, count);
    }
  printf("watch_count(): thread %ld Unlocking mutex.\n", my_id);
  pthread_mutex_unlock(&count_mutex);
  pthread_exit(NULL);
}

int main(int argc, char *argv[])
{
  int i, rc; 
  long t1=1, t2=2, t3=3;
  pthread_t threads[3];
  pthread_attr_t attr;

  /* Initialize mutex and condition variable objects */
  pthread_mutex_init(&count_mutex, NULL);
  pthread_cond_init (&count_threshold_cv, NULL);

  /* For portability, explicitly create threads in a joinable state */
  pthread_attr_init(&attr);
  pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
  pthread_create(&threads[0], &attr, watch_count, (void *)t1);
  pthread_create(&threads[1], &attr, inc_count, (void *)t2);
  pthread_create(&threads[2], &attr, inc_count, (void *)t3);

  /* Wait for all threads to complete */
  for (i = 0; i < NUM_THREADS; i++) {
   /* pthread_join(threads[i], NULL);*/
  }
  printf ("Main(): Waited and joined with %d threads. Final value of count = %d. Done.\n", 
          NUM_THREADS, count);

  /* Clean up and exit */
  pthread_attr_destroy(&attr);
  pthread_mutex_destroy(&count_mutex);
  pthread_cond_destroy(&count_threshold_cv);
  pthread_exit (NULL);

}

On Mac OS X 10.9(Apple LLVM version 5.1 (clang-503.0.40)), this will output:

            Starting watch_count(): thread 1
            Main(): Waited and joined with 3 threads. Final value of count = 0. Done.
            inc_count(): thread 2, count = 1, unlocking mutex
            inc_count(): thread 3, count = 2, unlocking mutex
            watch_count(): thread 1 Count= 2. Going into wait...
            watch_count(): thread 1 Condition signal received. Count= 2
            watch_count(): thread 1 Updating the value of count...
            watch_count(): thread 1 count now = 127.
            watch_count(): thread 1 Unlocking mutex.
            inc_count(): thread 2, count = 128, unlocking mutex
            inc_count(): thread 3, count = 129, unlocking mutex
            inc_count(): thread 2, count = 130, unlocking mutex
            inc_count(): thread 3, count = 131, unlocking mutex
            inc_count(): thread 2, count = 132, unlocking mutex
            inc_count(): thread 3, count = 133, unlocking mutex
            inc_count(): thread 2, count = 134, unlocking mutex
            inc_count(): thread 3, count = 135, unlocking mutex
            inc_count(): thread 2, count = 136, unlocking mutex
            inc_count(): thread 3, count = 137, unlocking mutex
            inc_count(): thread 2, count = 138, unlocking mutex
            inc_count(): thread 3, count = 139, unlocking mutex
            inc_count(): thread 2, count = 140, unlocking mutex
            inc_count(): thread 3, count = 141, unlocking mutex
            inc_count(): thread 2, count = 142, unlocking mutex
            inc_count(): thread 3, count = 143, unlocking mutex
            inc_count(): thread 2, count = 144, unlocking mutex
            inc_count(): thread 3, count = 145, unlocking mutex

And on CentOS 5 and 6(gcc 4.1.2 and gcc 4.4.3 x86_64-redhat-linux), the output appears more randomly, sometimes it is:

            Main(): Waited and joined with 3 threads. Final value of count = 0. Done.
            Starting watch_count(): thread 1
            watch_count(): thread 1 Count= 0. Going into wait...
            inc_count(): thread 2, count = 1, unlocking mutex
            inc_count(): thread 2, count = 2, unlocking mutex
            inc_count(): thread 2, count = 3, unlocking mutex
            inc_count(): thread 2, count = 4, unlocking mutex
            inc_count(): thread 2, count = 6, unlocking mutex
            inc_count(): thread 2, count = 7, unlocking mutex
            inc_count(): thread 2, count = 8, unlocking mutex
            inc_count(): thread 2, count = 9, unlocking mutex
            inc_count(): thread 2, count = 10, unlocking mutex
            inc_count(): thread 2, count = 11, unlocking mutex
            inc_count(): thread 3, count = 5, unlocking mutex

and it hangs, and some times it gives:

            Main(): Waited and joined with 3 threads. Final value of count = 0. Done.
            Starting watch_count(): thread 1
            watch_count(): thread 1 Count= 0. Going into wait...
            inc_count(): thread 2, count = 1, unlocking mutex
            inc_count(): thread 2, count = 2, unlocking mutex
            inc_count(): thread 2, count = 3, unlocking mutex
            inc_count(): thread 2, count = 4, unlocking mutex
            inc_count(): thread 2, count = 5, unlocking mutex
            inc_count(): thread 2, count = 6, unlocking mutex
            inc_count(): thread 2, count = 7, unlocking mutex
            inc_count(): thread 2, count = 8, unlocking mutex
            inc_count(): thread 2, count = 9, unlocking mutex
            inc_count(): thread 3, count = 10, unlocking mutex
            inc_count(): thread 3, count = 11, unlocking mutex
            inc_count(): thread 3, count = 12  Threshold reached. inc_count(): thread 2, count = 13, unlocking mutex

and also hangs.

To summarize: On Mac OS X, if I do not sleep in inc_count and join the 3 threads in main, the watch_count will receive signal(which apparently is not signaled by any of the 2 inc_count thread) when count is 2, and pthread_cond_signal never gets called.

On Linux however, it will hangs at some point.

my question is: How does pthread_join influence the behavior of condition variable? Why does Mac OS X behave so differently from Linux?

What are you hoping to do by removing the `pthread_join()` call? Without the calls to `pthread_join()` the program is cleaning up resources that are used by the threads (the mutex and the condition variable). Once those are destroyed by the main thread, their use in the other threads becomes undefined behavior. — Michael Burr, Jul 21 '14 at 22:06

score 0 · Answer 1 · answered Jul 21 '14 at 20:27

Taking out the pthread_join() is probably a mistake. Having started the various threads, the pthread_join() allows those threads to complete before exiting. If you don't do this then as the program exits, all threads which haven't completed are unceremoniously terminated. What you observe happening will depend on how far the threads got before the axe fell.

The sleep() just slows things down so the mechanics are more obvious.

I suspect that what is wrong here is that pthread_cond_signal() has no effect if there is no waiter... the signal is not remembered. Conditions should be used together with some other state. When the state is updated, the related condition is used to signal that the state has changed, in case some thread is waiting for that. So, a wait involving a simple flag condition will be:

pthread_mutex_lock(mutex) ;
while (!whatever)
  pthread_mutex_wait(condition, mutex) ;
pthread_mutex_unlock(mutex) ;

Note the while -- pthread_cond_signal() is allowed to wake up more than one thread, so from the waiters perspective, being woken up does not guarantee that the state has changed, or is what is waited for.

To signal this (where we are assuming that whatever is initialises false, elsewhere):

pthread_mutex_lock(mutex) ;
whatever = true ;
pthread_mutex_signal(condition) ;
pthread_mutex_unlock(mutex) ;

Noting, again, that the whatever flag is essential, because without it the waiter will wait forever if the signal is performed before the waiter thread gets to the wait !

Having said all that, I suggest that the watch_count() function is broken.

pthread_cond_wait not waked up correctly if not joined

1 Answers1