socket used in multithreading?

Question

I read code about socket used in multithread, the code is as follows:

function main

int main(void)
{
    int listenfd;
    int i = 0;

    /* check envirenment */
    if (check_env() == FALSE)
    {
        return 0;
    }

    /* get the bind port */
    listenfd = initserver(PORT); 

    if ( listenfd == -1 )
    {
        return 0;
    }

    /* initial the message queue. */
    /* and start to run ...*/
    initDatas(listenfd);

    /* start already.........*/
    /* make the main thread be a thread which recive the requst from 
     * client */
    fMsgIn((void *)&listenfd);

    return 0;
}

function initDatas

void initDatas(socketfd fd)
{
    int num_accept_req      = 5;
    int num_go              = 5;
    int num_getblg          = 5;

    /* control userbuf */
    init_userbuf();

    /* init the ctrlsockfd list */
    init_ctrlsockfd();

    /* run server */
    init_accept_req(fd, num_accept_req);

    /* get blog  */
    init_getblg(num_getblg);

    /* put blog */
//    init_pubblg(num_pubblg);

    /* get personal msg */
 //   init_getprsnalmsg(num_getprsnalmsg);
    /* pub personal msg */
  //  init_pubprsnalmsg(num_pubprsnalmsg);

    /*get followers */
   // init_getfollower(num_getfollower);

    /* set personal information */
    //init_setprsnalinfo(num_setprsnalinfo);

    /* send out dates ...*/
    init_msgout(num_go);
}

function init_accept_req

void init_accept_req(socketfd fd, int number_thread)
{
#ifdef DEBUG
    printf("\ninitial thread for accept request !\n");
    ASSERT(number_thread >= 1 && fd > 0);
#endif
    pthread_t *pid;
    pthread_attr_t attr;
    int i = 0;

    pid = Malloc_r(number_thread * sizeof(pthread_t));
    if ( pid == NULL )
        err_quit("malloc, in init_accept_req");

    pthread_attr_init(&attr);
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);

    for ( i = 0; i < number_thread; i++ )
    {
                            /* control accept requst */
        pthread_create(pid + i, &attr, (void *)fMsgIn, (void*)&fd);
    }
}

we can see that socket file descriptor listenfd is created by function initserver, and in function init_accept_req, multithread is created and linux socket function accept is called in the callback function of these thread, namely function fMsgIn, so my question is when multithreads are using the same socket fd, aren't there any conficts between these threads?(note that there are no synchronization primitives in these threads when call linux socket function accept)?

Answer 1

Answering the question:

Having multiple threads listing on the same socket does work, as for recent implementations accept() is thread save.

However one has to take care to check the outcome of all those parallel accept()s as multiple of tehm might return on a client attempting to connect, but only one accept() does this without an error.

Also one could argue this scenario is inefficient, due to those multiple returns.

---

However these calls

for ( i = 0; i < number_thread; i++ )
{
  pthread_create(pid + i, &attr, (void *)fMsgIn, (void*)&fd);
}

to create threads are potential killers, as they pass down to the thread function a reference to a variable local to

void init_accept_req(socketfd fd, int number_thread);

namely fd.

As soon as init_accept_req() has returned, fd is not valid anymore, nor is what the references, which had been passed to the thread functions, are pointing to.

To fix this pass a reference to the listening socket all the way down like so:

void init_accept_req(socketfd * pfd, int number_thread)
{
  [...]

  for ( i = 0; i < number_thread; i++ )
  {
                        /* control accept requst */
    pthread_create(pid + i, &attr, (void *)fMsgIn, (void*) pfd);
  }
}

void initDatas(socketfd * pfd)
{
  [...]

  init_accept_req(pfd, num_accept_req);

  [...]

int main(void)
{
  int listenfd;

  /* initial the message queue. */
  /* and start to run ...*/
  initDatas(&listenfd);

  [...]

Using this approach one only has to make sure main() does end (so that the listening socket listenfd stays valid) as long any of the accepting thread are doing their job.

A solution a bit dirty would be to misuse the thread function's void * typed user-data argument as int and pass down the socket descriptor by value like so:

pthread_create(pid + i, &attr, (void *)fMsgIn, (void*) fd);

Not nice, but feasable as long as sizeof(void*) isn't smaller then sizeof(int).

Answer 2

Normally it's a bad idea to operate on one socket at from more than one thread without synchronisation, though it is not prohibited. For instance, you can have one thread reading, and one thread writing. But if you try to have both threads reading, or both threads writing, you may not get a sensible result without any synchronisation. Similarly if you close() in one thread without synchronisation, you will run into trouble as the other thread will may end up accessing a non-existent FD.

As far as I understand it in the scheme you are trying to achieve, do have some synchronization of sorts. When you accept in your main thread, you are given a new fd which you then pass to the child thread whose responsibility it is. Your main thread is guaranteed to do nothing with that fd after the accept has taken place (and more importantly after the pthread_create() has been entered). The code sample you've posted is not complete, so it's hard to tell if you are actually achieving this (I can't see, for instance, where accept is being called).

Also Stephens' book on advanced UNIX programming is an invaluable resource for this sort of thing; it carries examples of many forked, non-forked and threaded servers and clients.