Implementing a user level thread library - return value from makecontext

Question

I've seen a few questions on user thread libraries, but none that seem to answer my question. I am able to create threads, run them, cancel them, and exit from them. What I cannot do for some reason is get a thread to return data.

When I initialize my thread library, I set my exit thread context as follows:

getcontext(&threadEnd);
threadEnd.uc_stack.ss_sp = (char *)malloc(SIGSTKSZ);
threadEnd.uc_stack.ss_size = SIGSTKSZ;
makecontext(&threadEnd, (void *) thread_exit, 1, &ReturnValue);

I create a thread and assign it as follows:

thread->myContext.uc_stack.ss_sp = (char *) malloc(SIGSTKSZ);
thread->myContext.uc_stack.ss_size = SIGSTKSZ;
thread->myContext.uc_link = &threadEnd;

When the function returns and thread_exit() is called:

    void thread_exit(void* retval){
    int* test;
    if (retval != NULL)
    {
        test = (int*) retval;
        printf("Returned value: %i\n", *test);
        fflush(stdout);
    }

The printout is always "Returned value: 0"

The called function is returning a pointer to an integer.

What am I doing wrong?

Answer 1

If you step through your program in GBD makecontext dose not save the return for the function used to make the context.

Example from my experiments: (observe the rax register):

at return statement:

thread1 (arg=0x1) at test_create_join.c:14
14      return (void *)11;
Value returned is $1 = 19
(gdb) info registers
rax            0x13 19
---

after return:

(gdb) step
15  }
(gdb) info registers
rax            0xb  11

inside context switch:

__start_context () at ../sysdeps/unix/sysv/linux/x86_64/__start_context.S:32
32  ../sysdeps/unix/sysv/linux/x86_64/__start_context.S: No such file or directory.
(gdb) info registers
rax            0xb  11

you can see for few instructions the return value is preserved but after few steps it becomes 0. Obviously its specific to x86_64 arch but I presume it might be same to most arch (ie the behavior of makecontext)

Now if you need the return value to your thread function you can go another way about it. Just create a thread handler for running your threads and use the handler for making new contexts. Here you can get the return for the functions you want to run as threads and save it in your thread control block structure for later use.

typedef struct {
    thread_start_routine start;
    void *arg;
} entry_point;

static void thread_runner(void *args) {
    exit_critical_section();

    entry_point *entry = (entry_point *) args;

    // run the thread and get the exit code
    current->retcode = entry->start(entry->arg);
}

Answer 2

You need some simple synchronization for waiting on one thread. The worker thread sets the variable test and signals.

// Pseudo code
// worker thread
retval = &VALUE; // retval is pointer?
SignalEvent( hEvent );

And the main thread:

// Pseudo code
// main thread
int* test;

hEvent = CreateEvent();

WaitOnEvent( hEvent );

if (retval != NULL)
{
    test = (int*) retval;
    printf("Returned value: %i\n", *test);
    fflush(stdout);
}

And with many threads you need to decide whether you wait to simply on any of them or to any but until they all signaled or you want to wait until all signaled and then check on value. Also we usually consider whether or not the thread that waits on events does some idle looping while waiting and possibly doing some other job (a bit more complicated case).

Should you use C++ I could've provided an example with C++ 11 condition variable but with C it is usually either POSIX or Win32 API but we can use pseudocode. Win32 SetEvent and WaitForSingleObject. This POSIX topic pretty much covers condition variables for you.

Mind that with threads finishing one thread is also triggering the signal so in Win32 you can wait directly on thread handle but this approach is maybe not portable. Also threads and their synchronization are not part of C language standard so it only makes sense to apply to certain OS that provides both.

Answer 3

First of all, when supplying arguments to makecontext, last one should always be NULL:

makecontext(&threadEnd, (void *) thread_exit, 1, &ReturnValue, NULL);

I had the same problem, I tackled it a bit differently, namely instead of using uc_link I stored start_routine, i.e. func and arg inside of thread structure and used a wrapper function: a thread_runner that actually call a thread's function and stored the return value:

makecontext(&thread->ctx, (void *) thread_runner, 1, thread, NULL);

where thread runner is:

void thread_runner(thread_t *thread){
  void **retval = thread->func(thread->arg);
  if(retval != NULL){
    thread_exit(*retval);
  } else {
    thread_exit(NULL);
  }
}

Answer 4

We are not supposed to pass pointers in the parameter list of makecontext(). They must be integers as specified in the manual:

the function func is called, and passed the series of integer (int) arguments that follow argc; the caller must specify the number of these arguments in argc.

On some architectures, integers have the same size as pointers (e.g. 32 bits) but on others the pointer is 64 bits whereas the integer is 32 bits long. On recent GLIBC/Linux x86_64 architectures, the parameters are stored as "long long" integers in the context. And so, this may work as this makes the parameters stored as 64 bits values (compatible with the size of pointers) but this is not portable. So, this may explain why you don't get a correct value passing the "&ReturnValue" pointer.