1

I'm trying to run QEMU's user mode emulator as a thread in a larger program that I'm writing. I've modified the linux-user/main.c file so that the standard int main(int argc, char **argv, char **envp function is now called void *qemu_user_mode_func(void *arg). I've also added pthread_exit(NULL) to the end of that function, as is standard practice for pthreads (or so I've been told).

However, when I try to run a second thread that contains my own test function (shown below in void *test_func(void *arg)), the process exits before the second thread completes, even with a call to pthread_join(tid), which I've read blocks the calling thread until thread tid returns. Does QEMU's user mode emulation exit in such a way that would prevent pthread_join from exiting, or am I just using threads wrong?

Here's my code (not including the bulk of qemu_user_mode_func):

void *qemu_user_mode_func(void *arg)
{
    thread_data_t *thread_data;
    int argc;
    char **argv;
    char **envp;

/** QEMU's normal code **/

    //return 0;
    pthread_exit(NULL);
}

void *test_func(void *arg) {
    struct timespec time;
    time.tv_sec = 7;
    time.tv_nsec = 0;

    nanosleep(&time, NULL);

    printf("hello, world - from a thread\n");
    pthread_exit(NULL);
}

int main(int argc, char**argv, char **envp) {
    //Initialize variables to create thread
    int rc;
    pthread_t threads[2];
    thread_data_t main_args;

    main_args.tid = 1;
    main_args.argc = argc;
    main_args.argv = argv;
    main_args.envp = envp;

    //Create thread
    if ((rc = pthread_create(&(threads[0]), NULL, test_func, NULL))) {
        fprintf(stderr, "error: pthread_create, rc: %d\n", rc);
        return EXIT_FAILURE;
    }

    if ((rc = pthread_create(&(threads[1]), NULL, qemu_user_mode_func, (void *)&main_args))) {
        fprintf(stderr, "error: pthread_create, rc: %d\n", rc);
        return EXIT_FAILURE;
    }

    //Wait for thread to finish, then terminate process
    for (rc = 0; rc < 2; rc++) {
        pthread_join(threads[rc], NULL);
    }

    return 0;
}

EDIT: I've discovered in the void cpu_loop(CPUX86State *env) function that when the emulated program reaches its conclusion, QEMU calls syscall 231, which is sys_exit_group (as per 1). So I'm guessing this syscall is terminating the entire process that I'm running. I'd appreciate any tips on how to get around that!

Lightness Races in Orbit
  • 378,754
  • 76
  • 643
  • 1,055
tonysdg
  • 1,335
  • 11
  • 32
  • Why don't you just `fork()` and call the qemu main function in the forked child? – caf Jun 27 '15 at 09:41
  • @caf - It has to do with how I'm using qemu - chiefly, I'm working on utilizing the dynamic binary translation system within it, known as the Tiny Code Generator. I'm trying to begin DBT partway through program execution - as in, a program starts on the native hardware and migrates to the virtual machine partway through execution. Since the text + data sections of the binary are shared between threads, any memory references to those sections are valid in both threads; all that changes is the stack (which is a whole other beast). I agree that `fork` is the better solution usually! – tonysdg Jun 29 '15 at 14:43

2 Answers2

0

If you turn a complicated preexisting application into thread there are going to be issues. One is that the application can call exit or its variants which will terminate your entire program. There are numerous other issues that could be causing a problem. I would suggest using gdb to determine what is making your program exit.

Community
  • 1
  • 1
missimer
  • 4,022
  • 1
  • 19
  • 33
  • I'm acutely aware of this haha - unfortunately, no other virtualization program performs (1) dynamic binary translation, and (2) has AARCH64 support (that I'm aware of). Fortunately, this turned out to have a simple (albeit ugly) fix. – tonysdg Jun 26 '15 at 17:28
0

Problem was solved by editing the following section in void cpu_loop(CPUX86State *env). I capture either sys_exit_group and sys_exit system calls before they are executed, and just return from the function instead.

Original:

void cpu_loop(CPUX86State *env)
{
    CPUState *cs = CPU(x86_env_get_cpu(env));
    int trapnr;
    abi_ulong pc;
    target_siginfo_t info;

    for(;;) {
        cpu_exec_start(cs);
        trapnr = cpu_x86_exec(env);
        cpu_exec_end(cs);
        switch(trapnr) {
        case 0x80:
            /* linux syscall from int $0x80 */
            env->regs[R_EAX] = do_syscall(env,
                                          env->regs[R_EAX],
                                          env->regs[R_EBX],
                                          env->regs[R_ECX],
                                          env->regs[R_EDX],
                                          env->regs[R_ESI],
                                          env->regs[R_EDI],
                                          env->regs[R_EBP],
                                          0, 0);
            break;
#ifndef TARGET_ABI32
        case EXCP_SYSCALL:
            /* linux syscall from syscall instruction */
            env->regs[R_EAX] = do_syscall(env,
                                          env->regs[R_EAX],
                                          env->regs[R_EDI],
                                          env->regs[R_ESI],
                                          env->regs[R_EDX],
                                          env->regs[10],
                                          env->regs[8],
                                          env->regs[9],
                                          0, 0);
            break;
#endif

Modified:

void cpu_loop(CPUX86State *env)
{
    CPUState *cs = CPU(x86_env_get_cpu(env));
    int trapnr;
    abi_ulong pc;
    target_siginfo_t info;

    for(;;) {
        cpu_exec_start(cs);
        trapnr = cpu_x86_exec(env);
        cpu_exec_end(cs);
        switch(trapnr) {
        case 0x80:
            /* linux syscall from int $0x80 */
            env->regs[R_EAX] = do_syscall(env,
                                          env->regs[R_EAX],
                                          env->regs[R_EBX],
                                          env->regs[R_ECX],
                                          env->regs[R_EDX],
                                          env->regs[R_ESI],
                                          env->regs[R_EDI],
                                          env->regs[R_EBP],
                                          0, 0);
            break;
#ifndef TARGET_ABI32
        case EXCP_SYSCALL:
            /* linux syscall from syscall instruction */
----> if ((env->regs[R_EAX] == __NR_exit_group) || (env->regs[R_EAX] == __NR_exit)) {
                return; 
            }
            env->regs[R_EAX] = do_syscall(env,
                                          env->regs[R_EAX],
                                          env->regs[R_EDI],
                                          env->regs[R_ESI],
                                          env->regs[R_EDX],
                                          env->regs[10],
                                          env->regs[8],
                                          env->regs[9],
                                          0, 0);
            break;
#endif
tonysdg
  • 1,335
  • 11
  • 32
  • Could you explain the changes that you made so that others can benefit in the future? – SevenBits Jun 26 '15 at 17:34
  • @SevenBits Yup - just edited the solution. Also switched to the original title - it think it captures the essence of the problem better. Thanks for the suggestion! – tonysdg Jun 26 '15 at 17:39