Thread Synchronization in Linux in C

Question

I am having issue synchronizing the threads so each thread can run one job first, then another thread start the same job, and so on. Below is my code:

#include <unistd.h>
#include <sys/types.h> 
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <string.h>
#include <semaphore.h>

void handler ( void *ptr );
sem_t mutex;

int worker = 2; // number of workers
int job = 4; // number of jobs for each worker

int main()
{

    int i = 0;
    pthread_t thread_a;

    sem_init(&mutex, 0, 1);

    for (i; i < worker; i++)
    {  
       int *n_workers = malloc(sizeof(*n_workers));  
       *n_workers = i;                   
       pthread_create (&thread_a, NULL, (void *) &handler, n_workers);
    }

    pthread_join(thread_a, NULL);

    sem_destroy(&mutex);

    pthread_exit(0);
}

void handler ( void *ptr )
{
    int x = *((int *) ptr);
    int i = 0;

    for (i; i < job; i++)
    {
       sem_wait(&mutex);
       printf("Worker %d: Doing Job %d\n", x, i);  
       sem_post(&mutex);
    }
}

The output is :

Worker 1: Doing Job 0
Worker 1: Doing Job 1
Worker 1: Doing Job 2
Worker 1: Doing Job 3
Worker 0: Doing Job 0
Worker 0: Doing Job 1
Worker 0: Doing Job 2
Worker 0: Doing Job 3

In the program, each worker has 4 jobs and there are 2 workers. The problem is that worker 1 does all 4 jobs at once and worker 0 does that all jobs after it. The ideal output would be this:

Worker 0: Doing Job 0
Worker 1: Doing Job 0
Worker 0: Doing Job 1
Worker 1: Doing Job 1
Worker 0: Doing Job 2
Worker 1: Doing Job 2
Worker 0: Doing Job 3
Worker 1: Doing Job 3

I am not sure where the issue is here, any help is greatly appreciated. Thanks

There isn't actually a problem here. Since the mutex/semaphore prevents more than one thread doing any work at once, they can't run in parallel; the scheduler is free to schedule the individual threads in any order. The order you're seeing is actually more efficient, because there is a cost to switching between threads (saving/restoring context; cache invalidation etc). Is it absolutely necessary for the jobs to complete in sequence? If so (given that you're not getting any parallelism) would doing it in one thread be a better option? — psmears, Mar 17 '15 at 15:53
@psmears thanks for your comment, yes, it would be necessary for each worker to start working in job n in sequence so they can complete the same job at the end of the program — Rain Man, Mar 17 '15 at 15:59
@RainMan Then why use threads? If you need that much control over the order, what benefit do you get from having more than one thread? Why do you want to force the most inefficient possible result? — David Schwartz, Mar 17 '15 at 15:59
@DavidSchwartz it is just for the concept, I know how to do this using specific number of threads, for example `thread_a, thread_b, thread_c` but I dont know how to do it this way — Rain Man, Mar 17 '15 at 16:04
BTW: you are overwriting the reference to the first thread created when creating the second thread. So the `pthread_join` only waits for the second thread, which might let the first thread access an already destroyed semaphore... — coltox, Mar 17 '15 at 16:09

score 1 · Answer 1 · answered Mar 17 '15 at 16:39

If you want to ensure that each thread does job[n] before job[n+1], you'll need to use barriers, either by implementing them using semaphores (for that you might want to consult The Little Book of Semaphores), or using pthread_barrier_t.

If you choose the latter, you should be able to achieve the effect with minimal modifications to your code:

#include <unistd.h>
#include <sys/types.h> 
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <string.h>
#include <semaphore.h>

void handler ( void *ptr );
pthread_barrier_t barrier;

int worker = 2; // number of workers
int job = 4; // number of jobs for each worker

int main()
{

    int i = 0;
    pthread_t thread_a;

    pthread_barrier_init(&barrier, NULL, worker);

    for (i; i < worker; i++)
    {  
       int *n_workers = malloc(sizeof(*n_workers));  
       *n_workers = i;                   
       pthread_create (&thread_a, NULL, (void *) &handler, n_workers);
    }

    pthread_join(thread_a, NULL);

    pthread_barrier_destroy(&barrier);

    pthread_exit(0);
}

void handler ( void *ptr )
{
    int x = *((int *) ptr);
    int i = 0;

    for (i; i < job; i++)
    {
       printf("Worker %d: Doing Job %d\n", x, i);  
       pthread_barrier_wait(&barrier);
    }
}

This way, each job[n] gets executed int worker = 2 times before work on job[n+1] starts. The order in which workers work on job[n] doesn't matter, so you might get different outputs, for example:

Worker 0: Doing Job 0
Worker 1: Doing Job 0
Worker 1: Doing Job 1
Worker 0: Doing Job 1
Worker 0: Doing Job 2
Worker 1: Doing Job 2
Worker 0: Doing Job 3
Worker 1: Doing Job 3

What's important is the rightmost column.

thanks got your answer, but the order of workers matters as well. I mean for future implementation, what if the worker can't start the job before the previous worker started working on it? in that case we would have problem — Rain Man, Mar 17 '15 at 17:11
That's probably a completely different problem then. Extrapolating, I can only suggest using queues, in [producer-consumer manner](https://en.wikipedia.org/wiki/Producer–consumer_problem); for example if you were making cakes, you'd have dough processors taking flour and water from appropriate queues, mixing them and putting dough in a dough queue. Then you'd need filling processors, which take dough from that queue and fill it, putting the result into a queue containing cakes - which you can then forward through your pipeline as well, for packaging or whatever. — vydd, Mar 17 '15 at 17:37
Internet Archive to the rescue again: [The Little Book of Semaphores](https://web.archive.org/web/20160304031330/http://www.greenteapress.com/semaphores/downey08semaphores.pdf) — natenho, Oct 26 '16 at 21:06

score 0 · Answer 2 · edited Mar 17 '15 at 16:14

It seems what you want strict alternation, which is not what you have implemented in your example. What your code instead does is making sure that the your threads' "jobs" are executed by at most one thread at a time.

Now every thread, when it gets scheduled gets a 'time-slice', a maximum amount of time that the thread may run. If it is not be forced to sleep by, e.g. a blocking operation it will not give up the CPU until this time-slice is completely consumed. That is exactly is happening here, as the sem_wait will never block for the first thread entering your critical region. You would have to use further synchronization to implement the strict alternation, if that is what you really want.

If you do not strictly need strict alternation, but just want to see the effect of multi-threading, you can also let the threads give up CPU voluntarily by calling int pthread_yield(void); after releasing the semaphore, to give other threads the possibility to run.

any suggestion on how to implement it so two thread's jobs can be executed? — Rain Man, Mar 17 '15 at 17:09
if I add " pthread_yield();" after the sem_post, then for 3 workers, the output is : http://pastebin.com/A0dgdQAT — Rain Man, Mar 17 '15 at 19:03

score 0 · Answer 3 · answered Jun 27 '21 at 17:04

    /* Output required (strictly in this order only)
     * Worker 0 is doing Job 0
     * Worker 1 is doing Job 0
     * Worker 0 is doing Job 1
     * Worker 1 is doing Job 1
     * Worker 0 is doing Job 2
     * Worker 1 is doing Job 2
     * Worker 0 is doing Job 3
     * Worker 1 is doing Job 3
     */

    #include<iostream>
    #include<semaphore.h>

    #define WORKERS 2
    #define JOBS 4

    sem_t ping, pong;

    void *worker0_func(void *junk)
    {
        for (int i = 0; i < JOBS; ++i)
        {
            sem_wait(&ping);
            std::cout << "Worker 0 is doing Job " << i << std::endl;
            sem_post(&pong);
        }
    }

    void *worker1_func(void *junk)
    {
        for (int i = 0; i < JOBS; ++i)
        {
            sem_wait(&pong);
            std::cout << "Worker 1 is doing Job " << i << std::endl;
            sem_post(&ping);
        }
    }

    int main()
    {
        pthread_t tid[WORKERS];
        sem_init(&ping, 0, 1); // Start Worker 0 immediately, all others will wait for signal
        sem_init(&pong, 0, 0);

        pthread_create(&tid[0], NULL, worker0_func, NULL);
        pthread_create(&tid[1], NULL, worker1_func, NULL);

        pthread_join(tid[0], NULL);
        pthread_join(tid[1], NULL);

        return 0;
    }

Thread Synchronization in Linux in C

3 Answers3