0

I am currently developing a x86-64 kernel from scratch for research purposes. The goal is to virtualize my own OS using Intels virtualization technology (VT-x). However I stumbled upon a xtremely strange problem which prevents me from continuing my project:

As soon after I enter the guest OS, the 32-bit entry point of my kernel needs to fetch the address of the 64-bit entry point for the kernel to jump to. This is done by retrieving the values of the kernel ELF header located in guest physical memory. However, at this point where the kernel ELF header is being retrieved, my OS throws an error saying that at the specified address there's no header:

Error: Kernel ELF header not found

To explain the image: the bottom half of the screen is the output of the host while the top half is the output of the guest.

Now I wondered why there is nothing even though I was pretty sure I placed the kernel ELF header at the specific address in guest physical memory (0x0019A000). Now a very important information to keep in mind: At this point where the error message comes up, paging has not been enabled yet. That means, just as described in the Intel documentation:

If CR0.PG = 0, each linear address is treated as a guest-physical address.

At this point paging is definitely not enabled since CR0 does not have bit 31 set. (Value of CR0: 0x60000011)

So I inspected the physical memory dump via Bochs:

enter image description here

Seems to be there, but the execution happens in the linear address space, to the instruction pointer is reading the instructions from the linear address space, and then there's the issue:

enter image description here

So my question is: What reasons do exist for the physical and the linear address space to differ from each other at just this one memory region while paging is disabled?

I checked:

  • The GDT (CS and DS setup for 32-bit, base = 0x0, limit = 0xfffff)
  • The selectors for the segments are set accordingly
  • The EPT used to map guest-physical addresses (GPA) to host-physical addresses (HPA): The address is definitely mapped, and even dumping the memory in the host-physical address space shows me that the kernel ELF header is there just as it should be
  • Paging is definitely not enabled, so the memdump should show the same content in memory at the same address
CRoemheld
  • 889
  • 7
  • 26
  • Yes is it enabled. The 32-bit entry point is located at an address > 1MB, and the screen obviously shows that the kernel is printing something which would not be possible if the A20 line wasn't enabled. – CRoemheld Oct 22 '18 at 23:09
  • I think you need to be more specific. You have three address spaces - physical address space, host linear address space and guest linear address space. When paging is disabled in the guest and enabled in host; you can expect `guest linear = host linear` and you can expect `host linear != physical` and `guest linear != physical`. – Brendan Oct 23 '18 at 02:16
  • @Brendan No we have four address spaces: host-physical, host-linear, guest-physical and guest-linear. And what you said is exactly what I already described in my question. I want to know what reasons do exist for the guest-physical and the guest-linear address space to be different at only this specific area in memory and everything else is both visible in the memory dump of the guest-physical and the guest-linear address space. – CRoemheld Oct 23 '18 at 08:48
  • @Brendon, host linear address space isn’t relevant while the guest is running. – prl Oct 23 '18 at 08:48
  • Guest linear and guest physical are the same, because paging is off, but guest physical and host physical may be different because EPT is enabled. – prl Oct 23 '18 at 08:50
  • @prl Okay maybe I was a little unclear about this. I used a block on continuous memory at host-physical address `0x08000000` (128MB) with the size of 128MB. Meaning, the guest OS has 128MB of available memory (minus the space occupied by itself, of course). The EPT maps the GPA `0x00000000 - 0x000B8000` to HPA `0x08000000 - 0x080B8000`, GPA `0x000B8000 - 0x000B9000` to HPA `0x000B8000 - 0x000B9000` and the rest of GPA `0x000B9000 - 0x08000000` to HPA `0x080B9000 - 0x10000000`. The mappings are correct and the memdump shows no wrong doings in the EPT. – CRoemheld Oct 23 '18 at 08:57
  • The memory dump from Bochs—is that Bochs running in the guest? – prl Oct 23 '18 at 08:59
  • @prl Yes and thats the issue here, guest-physical and guest-linear address space at this one memory region (GPA 0x0019A000) is different while paging is disabled. – CRoemheld Oct 23 '18 at 09:00
  • @prl Yep at the time the memdump was done, Bochs was operating in the guest. – CRoemheld Oct 23 '18 at 09:00
  • Is it possible that Bochs enables paging? I don’t fully understand the relationship between your guest code and Bochs. – prl Oct 23 '18 at 09:10
  • Is the VMM code yours also? That’s what I understood, but maybe I misinterpreted that too. – prl Oct 23 '18 at 09:12
  • Just to clarify one thing: it’s impossible for guest linear address and guest physical address to be different when paging is off, so even though that appears to be the symptom, we have to look for a different explanation. – prl Oct 23 '18 at 09:48
  • @prl Your comment about inspecting the host-physical address `0x0819A000` made me think because I actually assumed since the header is available in guest-physical address `0x0019A000`, that must mean it is definitely at `0x0819A000` in host-physical memory. Turns out it wasn't the case which made the confusion even bigger. But now I explicitely loaded the elf header at said address in host-physical memory and now it works. Thanks a lot. – CRoemheld Oct 23 '18 at 10:10

0 Answers0