-1

My single threaded HTTP Server works just fine, but I'm having trouble multithreading it. I know I am supposed to use pthreads, locks, and condition variables, but I can't get the logic set up properly. The trouble starts after listening to the server. Currently I have a struct that contains a client socket variable, a lock variable, a condition variable, and some variables necessary for parsing and storing headers. I create a struct array sized with the amount of threads, then create a pthread array sized with the amount of threads. I go into a while(1) loop which goes into a for loop and iterates through all the threads accepting each connection, calling pthread_create and passing them to my handle connections function, then closing the client socket. My handle connections then does the request handling that my single threaded http server did (reading, parsing, processing, constructing), then returns NULL. No request gets read when I run this using pthread_create, but if I run handle connections without the pthreads, it works just fine. And below I'll attach my code. Any help is appreciated

T-Wayne1
  • 35
  • 1
  • 6
  • You have posted *way* too much code, with *way* too incoherent a description of the problem. However, at a quick glance, you're `close()`ing your socket immediately after `pthread_create()`, without waiting for the thread to finish. – EOF May 15 '20 at 18:49
  • @EOF idk how much code to post on here tbh. I try posting only relevant code and people say its not enough. I try posting whole code on here and they say its too much :(. And when does the thread finish? I thought after returning to the main from the function its finished? – T-Wayne1 May 15 '20 at 19:09
  • A thread finishes when it either calls `pthread_exit()`, or when returning from the function that the thread started executing initially (the one passed to `pthread_create()`). But I was talking about the *work* the thread was supposed to do. That work will necessarily have finished when the thread does, but it might finish sooner (if the thread does not immediately terminate, like in a thread pool). You wait for neither. – EOF May 15 '20 at 19:31
  • Not to worry. IMO, you posted just the _right_ amount of code. Although, I only changed a small amount for my answer, I _needed_ to see all of it. That's because I examined the functions that I did _not_ change to see if they needed mutex locking, which they didn't. But, they _could_ have, _if_ there was more interaction between them. – Craig Estey May 15 '20 at 20:16

1 Answers1

0

Thank you for commenting so well ...

Okay, I coded up, but not tested the changes.

Your loop is inherently single threaded, so a bit of refactoring is in order

You have to scan for an unused thread control slot after doing accept.

You have to pthread_join completed/done threads [from any prior invocations].

The thread function has to close the per-client socket [not main thread]

You need a global (file scope) mutex.

I've coded it up, but not tested it. I put #if 0 around most of what I clipped out and #if 1 around new code.

Note that number of simultaneous connections [second arg to listen], herein 5 has to be less than or equal to threadNum. Although I didn't do it, I'd just do listen(...,threadNum) instead of hardwiring it.


Here's the short code with just the relevant changes:

#if 1
pthread_mutex_t global_mutex = PTHREAD_MUTEX_INITIALIZER;
#endif

struct threadObject {
    char method[5];                     // PUT, HEAD, GET. HEAD==4 letters+null terminator
    char filename[28];                  // what is the file we are worried about. Max 27 ASCII characters (NULL terminated on 28)
    char httpversion[9];                // HTTP/1.1
    ssize_t content_length;             // example: 13
    uint16_t status_code;               // status code for the request
    char buffer[BUFFER_SIZE];           // buffer to transfer data
    char rest_of_PUT[BUFFER_SIZE];      // incase client send part of PUT message in header
    int client_sockd;
    pthread_mutex_t *dispatch_lock;
    const pthread_cond_t *job_pool_empty;
    // pthread_mutex_t* log_lock;
    // const pthread_cond_t* log_pool_empty;
    pthread_mutex_t *read_write_lock;
    pthread_cond_t *file_list_update;
    // JobQueue* job_pool;
    // LogQueue log_pool;
    // bool is_logging;

#if 1
    pthread_t tsk_threadid;
    int tsk_inuse;
    int tsk_done;
#endif
};

void *
handle_connections(void *ptr_thread)
{
    // create a mutual exclusion to lock out any other threads from the function
    // pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
    // pthread_mutex_lock(&mutex);

    // operations go here
    struct threadObject *thread = (struct threadObject *) ptr_thread;

    // reset message after each loop
    memset(thread->buffer, '\0', BUFFER_SIZE);
    memset(thread->method, '\0', 5);
    memset(thread->filename, '\0', 28);
    memset(thread->httpversion, '\0', 9);
    thread->content_length = 0;
    thread->status_code = 0;
    memset(thread->rest_of_PUT, '\0', BUFFER_SIZE);

    // read message
    if (read_http_response(thread) == true) {
        // process message
        process_request(thread);
    }

    // construct a response
    construct_http_response(thread);

    // unlock the function
    // pthread_mutex_unlock(&mutex);

#if 1
    close(thread->client_sockd);

    pthread_mutex_lock(&global_mutex);
    thread->tsk_done = 1;
    pthread_mutex_unlock(&global_mutex);
#endif

    return NULL;
}

int
main(int argc, char **argv)
{
    // Create sockaddr_in with server information
    if (argc < 2) {
        perror("No arguments passed\n");
        return -1;
    }
    // make sure port number is above 1024 and set the port # to it
    if (atoi(argv[1]) < 1024) {
        return 1;
    }
    char *port = argv[1];

    // parse the command line args for options -l and -N. -l specifies it will use a log and the following parameter is the filename. -N specifies the number of threads it will use and the following parameter will be a number
    int opt;
    uint8_t threadNum = 1;
    char *logName = NULL;

    while ((opt = getopt(argc - 1, argv + 1, "N:l:")) != -1) {
        if (opt == 'N') {
            threadNum = atoi(optarg);
        }
        else if (opt == 'l') {
            logName = optarg;
        }
    }

    struct sockaddr_in server_addr;

    memset(&server_addr, 0, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(atoi(port));
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    socklen_t addrlen = sizeof(server_addr);

    // Create server socket
    int server_sockd = socket(AF_INET, SOCK_STREAM, 0);

    // Need to check if server_sockd < 0, meaning an error
    if (server_sockd < 0) {
        perror("socket");
        return 1;
    }

    // Configure server socket
    int enable = 1;

    // This allows you to avoid: 'Bind: Address Already in Use' error
    int ret = setsockopt(server_sockd, SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(enable));

    if (ret < 0) {
        return EXIT_FAILURE;
    }

    // Bind server address to socket that is open
    ret = bind(server_sockd, (struct sockaddr *) &server_addr, addrlen);
    if (ret < 0) {
        return EXIT_FAILURE;
    }

    // Listen for incoming connections
    ret = listen(server_sockd, 5);      // 5 should be enough, if not use SOMAXCONN
    if (ret < 0) {
        return EXIT_FAILURE;
    }

    struct threadObject thread[threadNum];

    // Connecting with a client
    struct sockaddr client_addr;
    socklen_t client_addrlen = sizeof(client_addr);

    // create a pthread array of size (number of threads). specify this will be using the handle connections function. join the threads together
#if 0
    pthread_t thread_id[threadNum];
#endif

#if 1
    struct threadObject *tsk = NULL;
    int tskidx;

    // clear out the thread structs
    for (tskidx = 0; tskidx < threadNum; tskidx++) {
        tsk = &thread[tskidx];
        memset(tsk,0,sizeof(struct threadObject));
    }

    while (true) {
        // accept connection
        int client_sockd = accept(server_sockd, &client_addr, &client_addrlen);

        pthread_mutex_lock(&global_mutex);

        // join any previously completed threads
        for (tskidx = 0; tskidx < threadNum; tskidx++) {
            tsk = &thread[tskidx];
            if (tsk->tsk_done) {
                pthread_join(tsk->tsk_threadid,NULL);
                tsk->tsk_inuse = 0;
                tsk->tsk_done = 0;
            }
        }

        // find unused task slot
        for (tskidx = 0; tskidx < threadNum; tskidx++) {
            tsk = &thread[tskidx];
            if (! tsk->tsk_inuse)
                break;
        }

        memset(tsk,0,sizeof(struct threadObject));
        tsk->client_sockd = client_sockd;
        tsk->tsk_inuse = 1;

        pthread_mutex_unlock(&global_mutex);

        // fire in the hole ...
        pthread_create(&tsk->tsk_threadid, NULL, handle_connections, tsk);
    }
#endif

#if 0
        for (int i = 0; i < threadNum; i++) {
            printf("\n[+] server is waiting...\n");
            thread[i].client_sockd = accept(server_sockd, &client_addr, &client_addrlen);
            handle_connections(&thread[i]);
            // pthread_create(&thread_id[i], NULL, handle_connections, &thread[i]);
            printf("Response Sent\n");
            // close the current client socket
            close(thread[i].client_sockd);
        }
    }
#endif

    return EXIT_SUCCESS;
}

Here's the complete code [just in case I clipped out too much]:

#include <sys/socket.h>
#include <sys/stat.h>
#include <stdio.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <fcntl.h>
#include <unistd.h>                     // write
#include <string.h>                     // memset
#include <stdlib.h>                     // atoi
#include <stdbool.h>                    // true, false
#include <errno.h>
#include <sys/types.h>
#include <ctype.h>
#include <pthread.h>
#define BUFFER_SIZE 4096

#if 1
pthread_mutex_t global_mutex = PTHREAD_MUTEX_INITIALIZER;
#endif

struct threadObject {
    char method[5];                     // PUT, HEAD, GET. HEAD==4 letters+null terminator
    char filename[28];                  // what is the file we are worried about. Max 27 ASCII characters (NULL terminated on 28)
    char httpversion[9];                // HTTP/1.1
    ssize_t content_length;             // example: 13
    uint16_t status_code;               // status code for the request
    char buffer[BUFFER_SIZE];           // buffer to transfer data
    char rest_of_PUT[BUFFER_SIZE];      // incase client send part of PUT message in header
    int client_sockd;
    pthread_mutex_t *dispatch_lock;
    const pthread_cond_t *job_pool_empty;
    // pthread_mutex_t* log_lock;
    // const pthread_cond_t* log_pool_empty;
    pthread_mutex_t *read_write_lock;
    pthread_cond_t *file_list_update;
    // JobQueue* job_pool;
    // LogQueue log_pool;
    // bool is_logging;

#if 1
    pthread_t tsk_threadid;
    int tsk_inuse;
    int tsk_done;
#endif
};

//read in the header and store it in the appropriate places
bool
read_http_response(struct threadObject *thread)
{
    printf("\nThis function will take care of reading message\n");
    // how many bytes we're receiving from the header. also puts the message into the buffer
    ssize_t bytes = recv(thread->client_sockd, thread->buffer, BUFFER_SIZE, 0);

    // if nothing or too much gets sent in the header, return
    if (bytes <= 0 || bytes >= BUFFER_SIZE) {
        thread->status_code = 400;
        printf("Too long or nothing in here\n");
        return false;
    }

    // NULL terminate the last spot on the buffer
    thread->buffer[bytes] = '\0';
    // how many bytes we received
    printf("[+] received %ld bytes from client\n[+] response: \n", bytes);
    printf("those bytes are: %s\n", thread->buffer);

    // make a char pointer pointer to the buffer to easily traverse it and parse it into the right spots
    char *traverse = thread->buffer;

    // first stop. sgnals the beginning of the filename
    char *file = strstr(traverse, "/");

    // 2nd stop. signls the beginning of the HTTP version. only 1.1 is accepted
    char *http = strstr(traverse, "HTTP/1.1");

    // 3rd stop. Signals the beginning of the content length
    char *contlength1 = strstr(traverse, "Content-Length");
    char *chunked = strstr(traverse, "chunked");

    if (chunked != NULL) {
        printf("MESSAGE NOT A FILE PUT\n");
        thread->status_code = 403;
        return false;
    }
    // store the method
    sscanf(traverse, "%s", thread->method);
    printf("method:%s\n", thread->method);
    // if its not 1 of the 3 valid requests, throw 400 error
    if (strcmp(thread->method, "GET") != 0 &&
        strcmp(thread->method, "PUT") != 0 &&
        strcmp(thread->method, "HEAD") != 0) {
        thread->status_code = 400;
        printf("Invalid Method:%s\n", thread->method);
        return false;
    }

    // if the filename doesnt start with /, its invalid throw 400 error
    if (*file != '/') {
        thread->status_code = 400;
        printf("bad filename\n");
        return false;
    }
    // only store the filename portion after the required /
    traverse = file + 1;
    // to make sure the filename isnt too long
    uint8_t size_check = 0;

    // traverse filename until first whitespace
    while (*traverse != ' ') {
        // if any character in the filename isnt 1 of these, its invalid. throw 400 error
        if (!isalnum(*traverse) && *traverse != '_' && *traverse != '-') {
            // if theres no filename at all, throw a 404 error
            if (size_check == 0) {
                thread->status_code = 404;
                printf("No file specified\n");
                return thread->status_code;
            }
            thread->status_code = 400;
            printf("Invalid filename character:%c\n", *traverse);
            return false;
        }
        sscanf(traverse++, "%c", thread->filename + size_check++);
        // if the filename breaks the 27 character limit, return a 400 error
        if (size_check > 27) {
            thread->status_code = 400;
            printf("filename too long\n");
            return false;
        }
    }
    printf("filename:%s\n", thread->filename);
    // if HTTP/1.1 isnt given, throw a 400 error
    if (http == NULL) {
        printf("HTTP/1.1 400 Bad Request\r\n\r\n");
        thread->status_code = 400;
        return false;
    }
    traverse = http;

    // read in the http version until the first \r\n. this signals the end of the given version name
    sscanf(traverse, "%[^\r\n]s", thread->httpversion);
    printf("HTTP:%s\n", thread->httpversion);
    // if its not a put request, this is the end of the header. return
    if (strcmp(thread->method, "PUT") != 0) {
        return true;
    }

    // for put requests only. traverse until the beginning of the content length
    traverse = contlength1;
    // last stop. signals the end of a normal PUT header. if a client wants to put some of the message in the header, it gets stored after this
    char *end = strstr(traverse, "\r\n\r\n");

    // if theres no \r\n\r\n, the header is bad. return 400
    if (end == NULL) {
        printf("bad header\n");
        thread->status_code = 400;
        return false;
    }
    // traverse to the next digit
    while (!isdigit(*traverse)) {
        // if theres no next digit after "content length", the header is bad. return 400
        if (traverse == end) {
            printf("bad header\n");
            thread->status_code = 400;
            return false;
        }
        traverse++;
    }
    // set to traverse to be sure fit the entire content length. use size_check to traverse through
    char *temp = traverse;

    size_check = 0;
    // while its taking in digits, put them into the char array.
    while (isdigit(*traverse)) {
        sscanf(traverse++, "%c", temp + size_check++);
    }
    // convert the new string into numbers
    thread->content_length = atoi(temp);
    // if the content length is < 0 throw a 400 error
    if (thread->content_length < 0) {
        thread->status_code = 400;
        printf("bad content length:%ld\n", thread->content_length);
        return false;
    }
    // printf("Content Length:%ld\n", thread->content_length);

    // move +4 spots to get to the end of this. if its a normal PUT, this will be the last spot. If the client puts part of the message in the header, it goes after this
    traverse = end + 4;
    // put the rest of the header into a char array to append later. if theres nothing, itll do nothing
    strcpy(thread->rest_of_PUT, traverse);
    // printf("Rest of PUT:%s\n", thread->rest_of_PUT);

    // will only get here if status code is 0
    return true;
}

//process the message we just recieved
void
process_request(struct threadObject *thread)
{

    printf("\nProcessing Request\n");
    // server side file descriptor
    int fd;

    // if the method is PUT
    if (strcmp(thread->method, "PUT") == 0) {

        // open the file for read only to check if its already there or not to set proper status code
        fd = open(thread->filename, O_WRONLY);
        // if it doesnt exist, set 201 status code
        struct stat checkExist;

        if (stat(thread->filename, &checkExist) != 0) {
            thread->status_code = 201;
        }
        // if it exists, set 200 and overwrite
        else {
            struct stat fileStat;

            fstat(fd, &fileStat);
            // check write permission
            if ((S_IWUSR & fileStat.st_mode) == 0) {
                printf("MESSAGE NOT WRITEABLE PUT\n");
                thread->status_code = 403;
                return;
            }
            thread->status_code = 200;
        }
        // close it
        close(fd);

        // reopen it. this time for writing to or overwriting. if its there, overwrite it. if not, create it. cant use for status codes since it will always create a new file
        fd = open(thread->filename, O_WRONLY | O_CREAT | O_TRUNC);
        // printf("fd in process is:%d\n", fd);
        // if theres a bad fd, throw a 403
        if (fd < 0) {
            printf("ERROR\n\n");
            thread->status_code = 403;
            return;
        }

        // to check that the amount of bytes sent = the amount received
        ssize_t bytes_recv,
         bytes_send;

        // if theres no body, put an empty file on the server
        if (thread->content_length == 0) {
            bytes_send = write(fd, '\0', 0);
        }
        // if there is a body, put it onto the new file created on the server and make sure the received bytes = the sent ones
        else {
            ssize_t total = 0,
                len_track = thread->content_length;

            while (thread->content_length != 0) {

                bytes_recv = recv(thread->client_sockd, thread->buffer, BUFFER_SIZE, 0);
                bytes_send = write(fd, thread->buffer, bytes_recv);
                total += bytes_send;
                // if the received bytes != the sent byes, send a 500 error
                if (bytes_recv != bytes_send) {
                    thread->status_code = 500;
                    printf("Recieved != sent for put request\n");
                    return;
                }

                thread->content_length -= bytes_recv;

                // printf("Bytes read:%ld\nBytes sent:%ld\nMessage content length:%ld\n", bytes_recv, bytes_send, message->content_length);
            }

            // if the content length != bytes sent, throw a 403 error
            if (len_track != total) {
                thread->status_code = 403;
                printf("Content length != sent for put request\n");
                return;
            }

        }

        printf("Message status code:%d\n", thread->status_code);
        // close the fd
        close(fd);
        return;
    }

    // if the method is GET or HEAD
    else if (strcmp(thread->method, "GET") == 0 || strcmp(thread->method, "HEAD") == 0) {
        // open the file for reading only
        fd = open(thread->filename, O_RDONLY);
        // if bad fd, throw a 404
        struct stat fileStat;

        fstat(fd, &fileStat);
        // check read permission and if it exists
        if (((S_IRUSR & fileStat.st_mode) == 0) || stat(thread->filename, &fileStat) != 0) {
            printf("BAD GET\n");
            thread->status_code = 404;
            return;
        }
        else {
            thread->status_code = 200;
            thread->content_length = lseek(fd, 0, SEEK_END);
        }
        // close the fd
        close(fd);
        return;
    }

}

void
construct_http_response(struct threadObject *thread)
{
    printf("Constructing Response\n");
    // size 22 since the largest code is 21 characters + NULL
    char response[22];

    // 200=OK, 201=CREATED, 400=BAD REQUEST, 403=FORBIDDEN, 404=NOT FOUND, 500=INTERNAL SERVER ERROR
    if (thread->status_code == 200) {
        strcpy(response, "OK");
    }
    else if (thread->status_code == 201) {
        strcpy(response, "CREATED");
    }
    else if (thread->status_code == 400) {
        strcpy(response, "BAD REQUEST");
    }
    else if (thread->status_code == 403) {
        strcpy(response, "FORBIDDEN");
    }
    else if (thread->status_code == 404) {
        strcpy(response, "NOT FOUND");
    }
    else if (thread->status_code == 500) {
        strcpy(response, "INTERNAL SERVER ERROR");
    }
    else {
        printf("Bad response...\n");
        return;
    }

    dprintf(thread->client_sockd, "%s %d %s\r\nContent-Length: %ld\r\n\r\n", thread->httpversion, thread->status_code, response, thread->content_length);

    if (strcmp(thread->method, "GET") == 0 && thread->status_code == 200) {
        int fd = open(thread->filename, O_RDONLY);

        ssize_t total = 0,
            len_track = thread->content_length,
            bytes_recv,
            bytes_send;

        while (thread->content_length != 0) {

            bytes_recv = read(fd, thread->buffer, BUFFER_SIZE);
            bytes_send = send(thread->client_sockd, thread->buffer, bytes_recv, 0);
            if (bytes_recv != bytes_send) {
                thread->status_code = 500;
                close(fd);
                printf("Recieved != sent for GET request\nReceived:%ld\nSent:%ld\n", bytes_recv, bytes_send);
                dprintf(thread->client_sockd, "%s %d %s\r\nContent-Length: %ld\r\n\r\n", thread->httpversion, thread->status_code, response, thread->content_length);
                close(fd);
                return;
            }

            total += bytes_send;

            thread->content_length -= bytes_recv;

        }

        if (total != len_track) {
            thread->status_code = 403;
            printf("Content length != recvd for GET request\n");
            dprintf(thread->client_sockd, "%s %d %s\r\nContent-Length: %ld\r\n\r\n", thread->httpversion, thread->status_code, response, thread->content_length);
            close(fd);
            return;
        }
        close(fd);
    }
}

void *
handle_connections(void *ptr_thread)
{
    // create a mutual exclusion to lock out any other threads from the function
    // pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
    // pthread_mutex_lock(&mutex);

    // operations go here
    struct threadObject *thread = (struct threadObject *) ptr_thread;

    // reset message after each loop
    memset(thread->buffer, '\0', BUFFER_SIZE);
    memset(thread->method, '\0', 5);
    memset(thread->filename, '\0', 28);
    memset(thread->httpversion, '\0', 9);
    thread->content_length = 0;
    thread->status_code = 0;
    memset(thread->rest_of_PUT, '\0', BUFFER_SIZE);

    // read message
    if (read_http_response(thread) == true) {
        // process message
        process_request(thread);
    }

    // construct a response
    construct_http_response(thread);

    // unlock the function
    // pthread_mutex_unlock(&mutex);

#if 1
    close(thread->client_sockd);

    pthread_mutex_lock(&global_mutex);
    thread->tsk_done = 1;
    pthread_mutex_unlock(&global_mutex);
#endif

    return NULL;
}

int
main(int argc, char **argv)
{
    // Create sockaddr_in with server information
    if (argc < 2) {
        perror("No arguments passed\n");
        return -1;
    }
    // make sure port number is above 1024 and set the port # to it
    if (atoi(argv[1]) < 1024) {
        return 1;
    }
    char *port = argv[1];

    // parse the command line args for options -l and -N. -l specifies it will use a log and the following parameter is the filename. -N specifies the number of threads it will use and the following parameter will be a number
    int opt;
    uint8_t threadNum = 1;
    char *logName = NULL;

    while ((opt = getopt(argc - 1, argv + 1, "N:l:")) != -1) {
        if (opt == 'N') {
            threadNum = atoi(optarg);
        }
        else if (opt == 'l') {
            logName = optarg;
        }
    }

    struct sockaddr_in server_addr;

    memset(&server_addr, 0, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(atoi(port));
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    socklen_t addrlen = sizeof(server_addr);

    // Create server socket
    int server_sockd = socket(AF_INET, SOCK_STREAM, 0);

    // Need to check if server_sockd < 0, meaning an error
    if (server_sockd < 0) {
        perror("socket");
        return 1;
    }

    // Configure server socket
    int enable = 1;

    // This allows you to avoid: 'Bind: Address Already in Use' error
    int ret = setsockopt(server_sockd, SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(enable));

    if (ret < 0) {
        return EXIT_FAILURE;
    }

    // Bind server address to socket that is open
    ret = bind(server_sockd, (struct sockaddr *) &server_addr, addrlen);
    if (ret < 0) {
        return EXIT_FAILURE;
    }

    // Listen for incoming connections
    ret = listen(server_sockd, 5);      // 5 should be enough, if not use SOMAXCONN
    if (ret < 0) {
        return EXIT_FAILURE;
    }

    struct threadObject thread[threadNum];

    // Connecting with a client
    struct sockaddr client_addr;
    socklen_t client_addrlen = sizeof(client_addr);

    // create a pthread array of size (number of threads). specify this will be using the handle connections function. join the threads together
#if 0
    pthread_t thread_id[threadNum];
#endif

#if 1
    struct threadObject *tsk = NULL;
    int tskidx;

    // clear out the thread structs
    for (tskidx = 0; tskidx < threadNum; tskidx++) {
        tsk = &thread[tskidx];
        memset(tsk,0,sizeof(struct threadObject));
    }

    while (true) {
        // accept connection
        int client_sockd = accept(server_sockd, &client_addr, &client_addrlen);

        pthread_mutex_lock(&global_mutex);

        // join any previously completed threads
        for (tskidx = 0; tskidx < threadNum; tskidx++) {
            tsk = &thread[tskidx];
            if (tsk->tsk_done) {
                pthread_join(tsk->tsk_threadid,NULL);
                tsk->tsk_inuse = 0;
                tsk->tsk_done = 0;
            }
        }

        // find unused task slot
        for (tskidx = 0; tskidx < threadNum; tskidx++) {
            tsk = &thread[tskidx];
            if (! tsk->tsk_inuse)
                break;
        }

        memset(tsk,0,sizeof(struct threadObject));
        tsk->client_sockd = client_sockd;
        tsk->tsk_inuse = 1;

        pthread_mutex_unlock(&global_mutex);

        // fire in the hole ...
        pthread_create(&tsk->tsk_threadid, NULL, handle_connections, tsk);
    }
#endif

#if 0
        for (int i = 0; i < threadNum; i++) {
            printf("\n[+] server is waiting...\n");
            thread[i].client_sockd = accept(server_sockd, &client_addr, &client_addrlen);
            handle_connections(&thread[i]);
            // pthread_create(&thread_id[i], NULL, handle_connections, &thread[i]);
            printf("Response Sent\n");
            // close the current client socket
            close(thread[i].client_sockd);
        }
    }
#endif

    return EXIT_SUCCESS;
}
Craig Estey
  • 30,627
  • 4
  • 24
  • 48
  • Jesus Christ you're fast. Thank you so much. I've been stuck on this forever now and this helps so much – T-Wayne1 May 15 '20 at 19:43
  • You're welcome. I've done a number of multithread implementations for mission critical realtime commercial products, so I've "already made [most of ;-)] my mistakes", so to speak ... – Craig Estey May 15 '20 at 20:34
  • Also, you had already done much of the prep work, so I _could_ go fast. You commented well, had _working_ single threaded code, and _already_ had `threadObject` and had coded up everything to use it. Many OPs, when posting their first multithread question, don't have such a struct [and it has to be created and explained why it's needed]. And, I _had_ to work fast, because you were getting downvoted [unfairly, IMO] because I found your question to be a good one with the right amount of detail. I was afraid if I didn't get an answer up, the question might have gotten closed [again, unfairly] – Craig Estey May 15 '20 at 20:48
  • What is with all tbe join stuff? It´s a server, right? Why would any join() be needed? The client handler threads can just exit/terminate? – Martin James May 15 '20 at 21:34
  • @MartinJames No, unless you _explicitly_ run the thread as detacted [with option] (when you start or later), you have to join it. Just like doing `wait` if you `fork` – Craig Estey May 15 '20 at 22:37
  • @CraigEstey why? If the OS terminates a process, it does not care whether the thread is 'joinable' or not. In any case, I meant implicitly that the client<>server threads should run detached. Join() is a PITA that is best avoided. – Martin James May 16 '20 at 06:23
  • @MartinJames Are you kidding? From `man pthread_join`: Failure to join with a thread that is joinable (i.e., one that is not detached), produces a "zombie thread". Avoid doing this, since each zombie thread consumes some system resources, and when enough zombie threads have accumulated, it will no longer be possible to create new threads (or processes). – Craig Estey May 16 '20 at 07:01
  • What does this do if all the "task slot"s are `tsk_inuse` ? @T-Wayne1: in any case, would it be less work to run just `threadNum` threads as a "work pool" ? I would rename the `threadObject` to be a `taskObject` and run `threadNum` threads which loop: (1) pull the next `taskObject` off a queue of same, (2) service the connection, (3) return `taskObject` to a free queue -- then the main thread waits for a free `taskObject` before accepting a new connection, then dispatches same to the work queue. – Chris Hall May 16 '20 at 09:20