1

How to send external interrupts to guest when following enabled:
Use TPR shadow
Virtualize APIC access
APIC register virtualization
Virtual-interrupt delivery
Acknowledge external interrupts
External interrupts exit
Process posted-interrupt

I've tried vmcs guest interrupt status 0x810, but couldn't make it work correctly. My goal is to redirect external interrupts to guest.

prl
  • 11,716
  • 2
  • 13
  • 31
GoodDevGuy
  • 13
  • 4
  • Are you asking how the VMM can inject interrupts, or how an external interrupt can be sent to the guest without a VM exit? For the former, you set the VIRR bit for the vector in the virtual APIC page and set RVI to max(RVI, vector). For the latter, you set up the IOMMU to do posted interrupts. – prl Feb 13 '23 at 22:27
  • The close vote is incorrect, this absolutely is a question about software development. – prl Feb 13 '23 at 22:29
  • Thank you for answer, I'm asking how to inject interrupts, tried to set VIRR and max(RVI, vector) but didn't work. Do I need to clear SVI every time I set RVI? and what about EOI? – GoodDevGuy Feb 13 '23 at 22:53
  • @prl I tried everything I could think of but nothing solved my problem, but I noticed that SVI never cleared, for example: 1- 1st vmexit externalInterrupt vector 0x6C: Set vIRR bit and RVI to max(RVI,Vector) RVI=0x0 SVI=0x0 => RVI=0x6C SVI=0x0 2- 2nd vmexit externalInterrupt vector 0x6C: Set vIRR bit and RVI to max(RVI,Vector) RVI=0x0 SVI=0x6C => RVI=0x6C SVI=0x6C At this point no other vmexit externalInterrupt happen, and devices using interrupts like keyboard stopped working. Does this indicate that something wrong or I simply need to clear SVI everything I set RVI? – GoodDevGuy Feb 14 '23 at 04:22
  • Sounds like a missing EOI. – prl Feb 14 '23 at 04:28
  • @prl thank you, while waiting for you I will do more tests and keep this post updated. – GoodDevGuy Feb 14 '23 at 04:29
  • @prl Yes its EOI problem, setting vector EOI bitmap and send EOI within EOIexitHandler solved the problem, is it possible to fix it without EOI exit? I've tried set TMR to level and didn't work. – GoodDevGuy Feb 14 '23 at 06:44
  • It's always the EOI... – prl Feb 14 '23 at 07:14
  • @prl Yes did read the SDM, the problem solution was to use EOI vmexits to send EOI to host apic to keep receiving interrupts, what can I do to not have to send EOI in root to keep receiving interrupts? – GoodDevGuy Feb 14 '23 at 07:28
  • Yeah, sorry, I misunderstood what you wrote before. For some reason with the EOI exit disabled, the EOI isn't going out correctly, but when you take the EOI exit, and send the EOI in the handler, then it works. So we have to figure out what is different between those two. What is your code doing that the EOI virtualization doesn't do... – prl Feb 14 '23 at 07:31
  • Is this interrupt an MSI or coming through the IOAPIC? I guess 6c is the legacy timer (PIT)? Can you check if it is programmed as level or edge in the IOAPIC? – prl Feb 14 '23 at 07:36
  • I thought of vTMR and tried to set it to level but didn't help, but if EOI exits isn't required it must something missing here including or excluding vTMR. Must be IOAPIC because MSI isn't enabled at that time, and its the legacy timer. – GoodDevGuy Feb 14 '23 at 07:37
  • Here's my guess: probably the virtual EOI is fine; probably you aren't doing a physical EOI before injecting the interrupt, so the ISR bit in the physical APIC is still set? – prl Feb 14 '23 at 07:41
  • @prl Yes you are right, sending EOI in external interrupt vmexit handler solved it, tried it before and didn't because I tested it with MP, now with SP everything looks fine, only problem left is when having MP, identifying this issue and looks like its related to access/write exits, can keep asking here if unable to solve it? thank you again for your time and effort. – GoodDevGuy Feb 14 '23 at 07:58
  • Make sure you have a distinct physical page of memory for the virtual APIC page for each hardware thread. (The APIC access page address can be the same for all of them.) – prl Feb 14 '23 at 08:46
  • @prl when I use ept to map apicBase to apicAccess-page and set vmcs apicAccess to apicAccess-page even SP stop working, but if I set vmcs apicAccess to apicBase FEE00000 SP works but MP doesn't, for WriteExits I copy written from Vapic to apic, and for AccessExits -LRead: use value from apic to emulate instr. -LWrite: use PrevValue from Vapic emulate instr and write result to both Vapic and apic. Can keep using apicBase as vmcs apicAccess? and is Access/Write handling correct? – GoodDevGuy Feb 14 '23 at 10:29
  • @prl I solved it, the problem was reading from vApic when emulating access, I had to read from apic. – GoodDevGuy Feb 14 '23 at 10:40
  • I've never set the APIC access address to FEE00000, but I'm not sure why not. I suppose if it works, you should see no difference in behavior. Since you do see a difference in behavior, then maybe it isn't a good thing to do. – prl Feb 14 '23 at 10:41
  • You might not want to propagate writes from the virtual APIC to the physical APIC, because they may not be in the same state. This is obviously true if you have more than one guest, because each guest has a distinct APIC state. But even if you have only one guest, it still seems problematic to require the virtual and the physical to remain in sync. – prl Feb 14 '23 at 10:42
  • @prl I agree, but for now I have only one guest and wanted to do it the easiest way possible and with less code and later will improve it without getting confused by complex code that I'm not sure about most of it. – GoodDevGuy Feb 14 '23 at 10:54
  • @prl after resumed working on my hypervisor I noticed that I accidentally disabled APIC virtualization, so what I said about fix is completely wrong, you can't redirect guest writes to host APIC, instead I had to emulate every write, specially ICR write, basically must emulate INIT SIPI and use posted interrupts for fixed without writing to host APIC except for NMI and posted interrupts notification vector. – GoodDevGuy Feb 16 '23 at 15:41

1 Answers1

1

To inject an interrupt into the guest when virtual interrupt delivery is enabled, follow steps similar to those that the CPU performs when it processes a posted interrupt, as described in SDM volume 3, section 30.6, "Posted Interrupt Processing", steps 4, 5, and 6.

The steps are:

  1. If the interrupt being injected into the guest is due to receipt of a physical interrupt, send EOI to the physical APIC.
  2. Set the bit in VIRR corresponding to the interrupt vector being injected.
  3. Set RVI to the vector number, if it is greater than the prior value of RVI.

Evaluation of pending virtual interrupts will be performed by the CPU upon VM entry.

See also SDM volume 3, section 30.2, "Evaluation and Delivery of Virtual Interrupts".

prl
  • 11,716
  • 2
  • 13
  • 31