How to make QEMU only return to the main loop by executing one instruction at a time?

Question

I added -fPCI to the QEMU source file compilation option and added -shared to the final link command, so that QEMU has become a shared library that can be dynamically loaded. I started trying to understand QEMU from then on. I use dlopen to dynamically load qemu and use dlsym to search for functions in qemu. This is my code:

#include<iostream>
#include<dlfcn.h>
#include<stdint.h>

using namespace std;

int main(int argc,char* argv[],char* envp[])
{
    void* handle = dlopen("/home/jack/qemu/qemu-5.0.0/arm-softmmu/libqemu-system-arm.so",RTLD_NOW);
    if(handle == nullptr)
    {
        printf("%s\n",dlerror());
        return 0;
    }

    void    (* qemu_init             )(int,char**,char**);
    void    (* qemu_main_loop        )(void);
    void    (* qemu_cleanup          )(void);
    bool    (* main_loop_should_exit )(void);
    void    (* main_loop_wait        )(int);
    int64_t (* cpu_get_icount        )(void);
    int64_t (* cpu_get_icount_raw    )(void);
    int64_t (* cpu_icount_to_ns      )(int64_t);
    int64_t (* cpu_get_clock         )(void);
    int64_t (* cpu_get_ticks         )(void);

#define GET_SYMBOL_AND_CHECK(X) *((void**)(&X)) = dlsym(handle,#X);if(nullptr == X){printf("lost symbol: "#X"\n");return 0;}
    GET_SYMBOL_AND_CHECK(qemu_init);
    GET_SYMBOL_AND_CHECK(qemu_main_loop);
    GET_SYMBOL_AND_CHECK(qemu_cleanup);
    GET_SYMBOL_AND_CHECK(main_loop_should_exit);
    GET_SYMBOL_AND_CHECK(main_loop_wait);
    GET_SYMBOL_AND_CHECK(cpu_get_icount);
    GET_SYMBOL_AND_CHECK(cpu_get_icount_raw);
    GET_SYMBOL_AND_CHECK(cpu_icount_to_ns);
    GET_SYMBOL_AND_CHECK(cpu_get_clock);
    GET_SYMBOL_AND_CHECK(cpu_get_ticks);
#undef GET_SYMBOL_AND_CHECK

    char* _argv[]=
    {
        "qemu-system-arm",
        "-M",
        "vexpress-a9",
        "-nographic",
        "-kernel",
        "/home/jack/temp/u-boot-2015.01/u-boot",
        "-icount",
        "1",
        "-singlestep",
        "-S",
        "-s",
    };
    int _argc=sizeof(_argv) / sizeof(_argv[0]);

    qemu_init(_argc,_argv,envp);

    while(!main_loop_should_exit())
    {
        main_loop_wait(false);

        //my test code:
        int64_t icount_raw = cpu_get_icount_raw();
        int64_t icount = cpu_get_icount();
        int64_t ticks = cpu_get_ticks();
        int64_t clock = cpu_get_clock();

        printf("----------icount_raw: %jd\n",icount_raw);
        printf("----------icount: %jd\n",icount);
        printf("----------ticks: %jd\n",ticks);
        printf("----------clock: %jd\n",clock);
    }

    qemu_cleanup();
    dlclose(handle);
    return 0;
}

The output of the program is as follows:

----------icount_raw: 0
----------icount: 0
----------ticks: 0
----------clock: 27595
----------icount_raw: 0
----------icount: 0
----------ticks: 0
----------clock: 47394


U-Boot 2015.01 (May 25 2020 - 14:42:11)

DRAM:  128 MiB
WARNING: Caches not enabled
Flash: 128 MiB
MMC:   MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   smc911x-0
Warning: smc911x-0 using MAC address from net device

Warning: Your board does not use generic board. Please read
doc/README.generic-board and take action. Boards not
upgraded by the late 2014 may break or be removed.
Hit any key to stop autoboot:  2 ----------icount_raw: 60040125
----------icount: 120271139
----------ticks: 120271139
----------clock: 1001128004
----------icount_raw: 119738560
----------icount: 239668009
----------ticks: 239668009
----------clock: 2002239949
----------icount_raw: 180295711
----------icount: 360782311
----------ticks: 360782311
----------clock: 3003347066
----------icount_raw: 240405702
----------icount: 481002293
----------ticks: 481002293
----------clock: 4004446427
----------icount_raw: 300858002
----------icount: 601906893
----------ticks: 601906893
----------clock: 5005552419
----------icount_raw: 361297422
----------icount: 722785733
----------ticks: 722785733
----------clock: 6006625721
----------icount_raw: 420679210
----------icount: 841549309
----------ticks: 841549309
----------clock: 7007717838
----------icount_raw: 424900860
----------icount: 849992609
----------ticks: 849992609
----------clock: 7082080834
----------icount_raw: 424900883
----------icount: 849992655
----------ticks: 849992655
----------clock: 7082105752
----------icount_raw: 424900906
----------icount: 849992701
----------ticks: 849992701
----------clock: 7082120318
QEMU: Terminated

I modeled QEMU's main loop and wrote this while loop. I print the acquired data every time in the loop, and I find that icount_raw can indicate the number of instructions currently executed by the CPU. Regarding other data, I am still confused. When this program is running, the uboot program can run normally. I find that the frequency of printing data on the screen is about once every second. Each time icount_raw increases a lot. When I use gdb to remotely control the program to run When using the "si" command, icount_raw is incremented by 1 each time, which is what I want to achieve: each time QEMU executes only one instruction, it can return to the main loop. I want to know how to modify the code of QEMU so that every time QEMU executes an instruction, it can return to the main loop instead of using gdb's "si" command. In the future, I want to know how to control QEMU to return to the main loop after each execution of N instructions. This N can be set freely by me.I understand that QEMU's event loop is based on Glib, and I think my problem may require modifying the code that calls Glib in QEMU.

Answer 1

Trying to take the internals of QEMU and put them into a DLL is something that's entirely unsupported, and so you're on your own for trying to figure out how to fix bugs in it, I'm afraid.

In general the behaviour you describe is expected: QEMU compiles guest code into "translation blocks" which correspond to multiple guest instructions, and it then also tries to directly create jumps between these translation blocks where it can. This is important for performance: we don't want to return to the top level execution loop more often than we absolutely have to.

Some of these optimisations are controllable: in upstream QEMU, the -singlestep command line option means "put only one guest instruction in each TB", and the "-d nochain" option means "don't do the optimisation that links TBs together". Mostly it's useful for debugging purposes to do this kind of thing: the behaviour is more easily comprehensible and debug logs easier to read. The downside is that the performance goes through the floor.