Understanding QEMU usermode emulation memory map

Question

I'm running an i386 binary under QEMU user mode emulation. The process makes a system call:

ioctl(5, USBDEVFS_BULK, {129, 96, 5000, 0x080d6698})

The userspace address 0x080d6698 is the output buffer for the ioctl.

When I run QEMU with -d page, I can see that this address is within a writable memory range in the guest process:

page layout changed following mmap
start    end      size     prot
08048000-080d2000 0008a000 r-x
080d2000-080d4000 00002000 rw-
080d4000-080f7000 00023000 rw-  <-- here
3fffd000-3ffff000 00002000 rw-
3ffff000-40000000 00001000 r-x
40000000-40001000 00001000 ---
40001000-40801000 00800000 rw-

However, from the host operating system's perspective, the memory map is actually like this. The address 0x080d6698 is not mapped.

    00400000-00726000      r-xp 00000000 fd:00 506        /usr/local/bin/qemu-i386
    00736000-0073a000      r--p 00326000 fd:00 506        /usr/local/bin/qemu-i386
    0073a000-00751000      rw-p 0032a000 fd:00 506        /usr/local/bin/qemu-i386
    00751000-00888000      rw-p 00000000 00:00 0          [heap]
--- [address 0x080d6698 does not fall into any range] ---
    4081a000-48862000      ---p 00000000 00:00 0 
    48862000-488ec000      r--p 00000000 fd:00 3189086    binary
    488ec000-488ee000      rw-p 0008a000 fd:00 3189086    binary
    488ee000-48911000      rw-p 00000000 00:00 0 
    48911000-80817000      ---p 00000000 00:00 0 
    80817000-80819000      rw-p 00000000 00:00 0 
    80819000-8081a000      r--p 00000000 00:00 0 
    8081a000-8081b000      ---p 00000000 00:00 0 
    8081b000-8101b000      rw-p 00000000 00:00 0 
    8101b000-140819000     ---p 00000000 00:00 0 
ffffe8000000-ffffeffff000  rwxp 00000000 00:00 0 
ffffeffff000-fffff0000000  ---p 00000000 00:00 0 
fffff0000000-fffff0021000  rw-p 00000000 00:00 0 
fffff0021000-fffff4000000  ---p 00000000 00:00 0 
fffff777c000-fffff77fd000  rw-p 00000000 00:00 0 
fffff77fd000-fffff77fe000  ---p 00000000 00:00 0 
fffff77fe000-fffff7ffe000  rw-p 00000000 00:00 0 
fffff7ffe000-fffff7fff000  r--p 00000000 00:00 0          [vvar]
fffff7fff000-fffff8000000  r-xp 00000000 00:00 0          [vdso]
fffffffdf000-1000000000000 rw-p 00000000 00:00 0          [stack]

I believe I'm seeing the range 080d4000-080f7000 have guest_base = 0x4081a000 added to it to become 488ee000-48911000.

I'm not sure I'm looking in the right spot, but it appears that QEMU just zero-extends the address for a 32-bit guest on a 64-bit host, but it doesn't translate the address mappings.

In that case, shouldn't the memory map from QEMU's debug output basically match what Linux reports? Instead, they appear to be completely different.

score 1 · Answer 1 · answered Aug 03 '23 at 10:42

Yes, in user-mode, QEMU does not emulate a guest MMU. Guest virtual addresses are converted to host addresses by adding a fixed offset, with the aim of being able to fit the full 32-bit address space of a 32-bit guest into an "empty" area of the host process 64-bit address space. (For 64-bit guest on 64-bit host, this obviously won't work, and so the offset is generally 0.)

In your case the offset is indeed 0x4081a000. So these guest mappings:

08048000-080d2000 0008a000 r-x
080d2000-080d4000 00002000 rw-
080d4000-080f7000 00023000 rw-  <-- here

are these host mappings:

    48862000-488ec000      r--p 00000000 fd:00 3189086    binary
    488ec000-488ee000      rw-p 0008a000 fd:00 3189086    binary
    488ee000-48911000      rw-p 00000000 00:00 0

and these guest mappings:

3fffd000-3ffff000 00002000 rw-
3ffff000-40000000 00001000 r-x
40000000-40001000 00001000 ---
40001000-40801000 00800000 rw-

are these host mappings:

    80817000-80819000      rw-p 00000000 00:00 0 
    80819000-8081a000      r--p 00000000 00:00 0 
    8081a000-8081b000      ---p 00000000 00:00 0 
    8081b000-8101b000      rw-p 00000000 00:00 0

In the host mapping output you will also see other things that QEMU itself is mapping:

QEMU's own executable
QEMU's own stack, heap, etc
"no rwx" mappings in areas that correspond to the guest address space -- QEMU puts these in explicitly, so that they are reserved for the guest to use later. The mapping at 4081a000-48862000 is an example of this

So you can see that the memory mappings are not "completely different" -- the host one just has extra entries for QEMU's own purposes, and the guest entries within it are all at a fixed offset.

Thanks. So if the guest is not compiled for position-independent execution, how does it cope with the addresses being offset? For example in this case the guest is trying to make a syscall with the non-offset address which the kernel rejects. — rgov, Aug 04 '23 at 19:12

Understanding QEMU usermode emulation memory map

1 Answers1