1
bebc <mystery>:
 mov    3423441(%rip),%r11
 xor    (%rsp),%r11
 mov    %rdx,%r8
 mov    %rcx,%r9
 test   %r8,%r8
 jne    bee3
 mov    $0x6c16,%rdi
 mov    $0xc0f4,%rax
 vmwrite %rax,%rdi
 pushf

For the above instruction, which instruction can possibly cause an exit and which type of exit? Also what's the reason for exit? (Assume the page contains mystery function).The mystery function is running in CPL0.

ABC
  • 11
  • 3
  • 1
    The way I see it, it's just the obvious one: the `vmwrite`. And it causes a `vmwrite` type exit (code 25). – Jester Oct 28 '20 at 12:41
  • @Jester You mean to say `vmwrite` is the only instruction which can possibly cause an exit? – ABC Oct 28 '20 at 12:49
  • Technically a page fault could also cause a vm exit if it's configured to do so. Assuming the code itself is present and executable, that means instructions that access memory. – Jester Oct 28 '20 at 13:04
  • @Jester Thanks for the insights. Here the assumption made is that the page containing the mystery function (i.e the page containing addresses 0xb000 – 0xbfff) is present in memory (with all page tables properly configured). The mystery function is running in CPL0. So given this assumption which all instructions can possibly cause an exit? – ABC Oct 28 '20 at 13:17
  • As I said, `vmwrite` and any instruction that accesses memory. – Jester Oct 28 '20 at 13:24

1 Answers1

3

This answer is based on the Intel VMX extensions, AMD probably differ.
Furthermore, VM extensions had become quite vast and complex (as Oct 2020), so the answer may not suprise you:

They all can and cannot, depending on the VM configuration, cause a VM-exit (henceforward, an exit).

Depending on the context, probably only vmwrite will cause an exit but, again, VMCS shadowing (read: nested virtualization) has been supported for a while by all the mainstream VMMs, AFAICT, and so maybe neither that instruction will cause an exit.

The code seems the prolog a function conforming to the Windows ABI, that sets a stack cookie and eventually overwrite the host rip in the host state area.


bebc:       4c 8b 1d e5 57 30 00    mov    3168229(%rip),%r11

Can cause an exit if it faults (see section 25.2 of the Intel's manual):

Exceptions. Exceptions (faults, traps, and aborts) cause VM exits based on the exception bitmap (see Section 24.6.3). If an exception occurs, its vector (in the range 0–31) is used to select a bit in the exception bitmap. If the bit is 1, a VM exit occurs; if the bit is 0, the exception is delivered normally through the guest IDT.

If exiting windows are configured and the relative interrupt is not masked, before executing this instruction the CPU will generate an exit, though this may not count as an exit generated by the instruction itself.

If the “interrupt-window exiting” VM-execution control is 1, a VM exit occurs before execution of any instruction if RFLAGS.IF = 1 and there is no blocking of events by STI or by MOV SS (see Table 24-3).

If the “NMI-window exiting” VM-execution control is 1, a VM exit occurs before execution of any instruction if there is no virtual-NMI blocking and there is no blocking of events by MOV SS and no blocking of events by STI (see Table 24-3).

If the EPTs are used and they are misconfigured or an EPT-violation (i.e. to emulate a MMIO region) is detected, then an exit is induced.

Accesses using guest-physical addresses may cause VM exits due to EPT misconfigurations, EPT violations, and page-modification log-full events. An EPT misconfiguration occurs when, in the course of translating a guest- physical address, the logical processor encounters an EPT paging-structure entry that contains an unsupported value (see Section 28.2.3.1). An EPT violation occurs when there is no EPT misconfiguration but the EPT paging- structure entries disallow an access using the guest-physical address (see Section 28.2.3.2). A page-modifica- tion log-full event occurs when the logical processor determines a need to create a page-modification log entry and the current log is full (see Section 28.2.6).

 bec3:       4c 33 1c 24             xor    (%rsp),%r11

Ditto.

 bec7:       49 89 d0                mov    %rdx,%r8

This instruction cannot fault but otherwise the same exits apply.

 beca:       49 89 c9                mov    %rcx,%r9
 becd:       4d 85 c0                test   %r8,%r8
 bed0:       75 11                   jne    bee3
 bed2:       48 c7 c7 16 6c 00 00    mov    $0x6c16,%rdi
 bed9:       48 c7 c0 f4 c0 34 82    mov    $0xc0f4,%rax

Ditto for these.

 bee0:       0f 79 f8                vmwrite %rax,%rdi

Besides the pedantic exits applying to all instructions, this one can cause an exit if:

VMWRITE. The VMWRITE instruction causes a VM exit if any of the following are true:
— The “VMCS shadowing” VM-execution control is 0.
— Bits 63:15 (bits 31:15 outside 64-bit mode) of the register source operand are not all 0.
— Bit n in VMWRITE bitmap is 1, where n is the value of bits 14:0 of the register source operand. See Section 24.6.15 for details regarding how the VMWRITE bitmap is identified.

 bee3:       9c                      pushf

Some consideration for any other instruction accessing memory.


These are the instructions that always cause an exit:

  • INVEPT
  • INVVPID
  • VMCALL
  • VMCLEAR
  • VMLAUNCH
  • VMPTRLD
  • VMPTRST
  • VMRESUME
  • VMXOFF
  • VMXON

These instruction could case an exit:

  • CLTS
  • ENCLS
  • ENCLV
  • HLT
  • IN, INS/INSB/INSW/INSD, OUT, OUTS/OUTSB/OUTSW/OUTSD
  • INVLPG
  • INVPCID
  • LGDT, LIDT, LLDT, LTR, SGDT, SIDT, SLDT, STR
  • LMSW
  • MONITOR
  • MOV from CR3
  • MOV from CR8
  • MOV to CR0
  • MOV to CR3
  • MOV to CR4
  • MOV to CR8
  • MOV DR
  • MWAIT
  • PAUSE
  • RDMSR
  • RDPMC
  • RDRAND
  • RDSEED
  • RDTSC
  • RDTSCP
  • RSM
  • TPAUSE
  • UMWAIT
  • VMREAD
  • VMWRITE
  • WBINVD
  • WRMSR
  • XRSTORS
  • XSAVES

And finally, these are also other sources of exits:

  • Exceptions
  • Triple fault
  • External interrupts
  • Non-maskable interrupts (NMIs)
  • INIT signals
  • Start-up IPIs (SIPIs)
  • Task switches
  • VMX-preemption timer
  • interrupt-window exiting
  • NMI-window exiting
Margaret Bloom
  • 41,768
  • 5
  • 78
  • 124
  • Thanks for a detailed explanation. It would be great to learn more on whether the instruction exits if the pre-requisites are that the page containing the mystery function (i.e, the page containing addresses 0xb000 – 0xbfff) is present in memory (with all page tables properly configured). The mystery function is running in CPL0. Other assumptions to be ignored. Can you tell which all instructions exit, also the type of exit, considering the above scenario and state a short reason for the same. Thanks again for giving a detailed explanation of various environment scenarios. – ABC Oct 28 '20 at 18:14
  • Can `jne bee3` really fault, though? If taken, it's jumping into the same virtual page that it itself executes from. I don't think execution could reach the `jne` but then fault on the branch target. (And even if you did jump to a faulting address, it wouldn't truly be the `jne` that faulted, it would be code fetch from the new RIP, right? The fault address would be the `bee3`, not the `bed0` address of the `jne`. Of course if these are pre-relocation in a `.o`, then alignment relative to a 4k page isn't nailed down yet. But in a linked PIE, it is.) – Peter Cordes Oct 28 '20 at 18:59
  • `mov $0x6c16,%rdi` - that's an immediate source, not like the `xor (%rsp), %r11`. It's also silly and inefficient to use a 7-byte `mov $sign_extended_imm32, %r64` instead of 5-byte `mov $imm32, %r32`, unless this is un-linked kernel code and that's only a placeholder for an absolute address in the high 2GiB... Otherwise just the usual result of someone using an assembler like GAS instead of NASM but not optimizing manually for small constants. – Peter Cordes Oct 28 '20 at 19:04
  • @PeterCordes Does that mean jne bee3 will not cause fault, but bee3 will cause fault – ABC Oct 28 '20 at 19:14
  • @ABC: https://www.felixcloutier.com/x86/jcc#64-bit-mode-exceptions - JCC itself can only fault in 2 cases: `#GP(0)` if the target address is non-canonical (e.g. jump forward from `0x7ffffffffffe`), or `#UD` if the LOCK prefix is used. It can't page-fault (unless the memory containing the instruction bytes isn't executable, but then JCC hasn't even started executing). So yes, the jump itself can't fault, and AFAIK that means it can't VM-exit. In this case, the branch target is in the same page, so code-fetch from `(relocation<<12) + bee3` can't fault either. – Peter Cordes Oct 28 '20 at 19:19
  • The `push` instruction there could page-fault on the data load/store as it *executes*, but that wouldn't be `jne`'s fault. Code *fetch* from the target address will succeed (and that isn't truly the branch's fault anyway.) – Peter Cordes Oct 28 '20 at 19:24
  • @ABC: I guess it's possible that the page-table entry could be invalidated and the TLB entry that covers this page could have been evicted between fetching the `jne` and trying to fetch the `pushf` (needing a page walk, but the page walk fails), so it's possible for the `jne` to execute but code-fetch from `bee3` to fail, even without an interrupt coming in. For that to happen, this or another core would have to have changed a PTE from valid to invalid *without* using `invlpg`, which is already a bad idea. But the consequence is that the TLB entry keeps working until it's evicted. – Peter Cordes Oct 28 '20 at 19:27
  • @PeterCordes I think you are right, being on the same page of the target, should prevent `jne` to fault. Fixing that and the immediate thing (I rarely use AT&T). Thank you! – Margaret Bloom Oct 28 '20 at 20:01
  • @ABC Is this an homework? The only probable instruction to cause an exit is `vmwrite` because otherwise it would alter the VMM host memory. If nested virtualization is enabled, which often is, then even that instruction is virtualized without an exit. The exit reasons are listen on Appendix C, you'll find the VMWRITE exit code there. – Margaret Bloom Oct 28 '20 at 20:07
  • @MargaretBloom Thanks for the detailed explanation, could your list the reference to the documentation of instructions that always cause an exit and instructions could case an exit? – tristone Feb 02 '23 at 04:16