How to control number of threads by using pthread_join in C?

Question

I have created a C program which will read 20000 strings from a text file, and send it to other program. I have used a while to loop through this text file and create threads which will send that text to the other program. But I want only 4 threads to work. so I have used a counter and keep decrementing it, and an if condition to check the counter and when it will be set to 1 then I have called pthread_join for the previous threads. I want to first finish these 4 threads and then new 4 threads to pick up the new text file strings. But it is not working as I need. it processes only every 4th thread 4 times. and doesnt picks up all records from the text file.

Program:-

int Read_record()
{
    printf("Inside Read_record()\n");
    pthread_t threads;
    int rc;
    char l_record[300];
    int thNum=4;

    while(1){
        MEMSET(g_record);

        if(fgets(g_record,300,g_r_fp)==NULL){
            printf("End of File.\n");
            break;
        }else{
            //printf("%s",g_record);
            printf("%s",g_record);
            rc = pthread_create(&threads, NULL, &Get_report, (void *)g_record);

            if (rc){
                printf("ERROR; return code from pthread_create() is %d\n", rc);
                exit(-1);
            }

            thNum--;
        }

        if(thNum==0){ 
            pthread_join(threads, NULL); 
            thNum=4;
        }
    }

    return 0;
}

Input Text file contains: 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Out put comes:-

Inside Read_record()
1
2
3
4
Inside Get_report, wget 4

Inside Get_report, wget 4

Inside Get_report, wget 4

Inside Get_report, wget 4

5
6
7
8
Inside Get_report, wget 8

Inside Get_report, wget 8

Inside Get_report, wget 8

Inside Get_report, wget 8

9
10
11
12
Inside Get_report, wget 12

Inside Get_report, wget 12

Inside Get_report, wget 12

Inside Get_report, wget 12

13
14
15
16
Inside Get_report, wget 16

Inside Get_report, wget 16

Inside Get_report, wget 16

Inside Get_report, wget 16

17
18
19
20
Inside Get_report, wget 20

Inside Get_report, wget 20

Inside Get_report, wget 20

Inside Get_report, wget 20

End of File.

Desire Output:-

Inside Get_report, wget 1
Inside Get_report, wget 2
Inside Get_report, wget 3
Inside Get_report, wget 4

Inside Get_report, wget 5
Inside Get_report, wget 6
Inside Get_report, wget 7
Inside Get_report, wget 8

Inside Get_report, wget 9
Inside Get_report, wget 10
Inside Get_report, wget 11
Inside Get_report, wget 12.
and so on..

Please mind I want only 4 threads to create in system. not more than that.

You create four threads, but save the thread id to only one variable (overwriting each time you create a thread), so you will only join the last thread. — Some programmer dude, Sep 11 '12 at 07:28
Thanks Joachim. Do you want me to create 4 different thread variables, and then put a join for each thread ? like thread1. thread2. thread3, thread4. and then pthread_join(thread1,NULL), pthread_join(thread2,NULL), pthread_join(thread3,NULL), pthread_join(thread4,NULL) ? but would it be Multithreading ? — CodeCodeCode, Sep 11 '12 at 08:35
An array would probably be better, as you can then use e.g. `thNum` to index it on creation. — Some programmer dude, Sep 11 '12 at 08:40

Brady · Answer 1 · 2012-09-11T07:44:14.187

Sounds like what you need is a thread pool, whereby you create a pool of 4 threads and communicate the work to them via a queue. The main thread will read the file and enqueue jobs (object/struct with the line of text to process) on the queue. The threads will take a job off the queue, process the job, and then get another job.

The threads should not exit after processing just one job, but just loop until told to stop.

Minion91 · Answer 2 · 2012-09-11T08:07:13.113

1

There is something fundamentally wrong with the structure of your program. Here is what i think you want:

Have the main method read the strings
Have the main method create 4 threads (store them in an array or something)
Implement some form of shared memory to send the data to the 4 threads
You will probably need some form of synchronization.
after the work is done terminate and join on all 4 threads

hope this gets you on your way to the wonderful world of threading

This is in fact a thread pool as mentioned in the other post

And i found this wonderful tutorial on threads, i only looked through the index table but seems to have everything you need covered.

edited Sep 11 '12 at 08:07

answered Sep 11 '12 at 07:31

Minion91

1,911
12
19

however you want to call it :) The word is just borrowed from languages where this object is implemented in the standard libs – Minion91 Sep 11 '12 at 07:35
Your answer is completely correct, but the point was that it sounds very similar (possibly too similar) to the answer already posted. – Brady Sep 11 '12 at 07:37
Could be, although imho my answer is a little bit better to get a starting programmer started. And i'll add a link to the thread pool pattern, it's actually completely the same ^^ – Minion91 Sep 11 '12 at 07:40
I think there is a need to mention mutex (or any other synchronization) mechanism regarding the shared memory queue. – eyalm Sep 11 '12 at 07:49
That depends on the implementation. If all data transactions are initialized by the main thread and different pieces of shared memory are used there is no need for a locking mechanism. Although I see your point and will update my post in a minute. – Minion91 Sep 11 '12 at 07:55
Synchronization is absolutely necessary for a shared resource, either implicitly or explicitly, depending on the implementation used. – Brady Sep 11 '12 at 08:12
That is correct, but you don't need any locking mechanism for implicit synchronization, which is what i said – Minion91 Sep 11 '12 at 08:18
When I mention implicit, I mean its implemented internally in the queue. – Brady Sep 11 '12 at 08:28

How to control number of threads by using pthread_join in C?

2 Answers2