2

I've came across Advanced Virtual Interrupt Controller (AVIC) in the AMD64 Architecture Programming Manual (APM), Volume 2. Some bits were unclear to me, so I've quickly skimmed through the popular open-source hypervisors (Qemu/KVM and Xen, to name a few) sources to see how it is used together with AMD IOMMU. It appeared that none of them are using AVIC, and Bochs/Qemu don't emulate it either (there is an IOMMU emulation in Qemu, but it's for Revision 1 that doesn't virtualize interrupts).

So the two questions arise:

  1. Why the AVIC is so "unpopular"? Maybe it isn't broadly supported by the CPUs today on the market, or is it because all these hypervisors have a long history and already virtualize interrupts by themselves, so porting to AVIC isn't a top priority? (or have I just missed something in the sources?)

  2. [The original question] Is the Guest Virtual APIC Table Root Pointer in IOMMU's Device Table Entry a pointer to Physical APIC ID Table, as defined in APM Vol. 2, Sect. 15.29.2.3?

Thank you for the clarifications.

vsinitsyn
  • 113
  • 7

2 Answers2

2

Looks like the AVIC is not supported even in 16h family processors (the "Preliminary BIOS and Kernel Developer’s Guide (BKDG) for AMD Family 16h Models 00h-0Fh (Kabini) Processors" says that CPUID Fn8000_000A_EDX[AVIC] value is 0). This is likely the reason it is not implemented in the hypervisors either.

It also appeared that the second question was due to me using a different APM and IOMMU specification revisions. When I got the compatible ones (APM Rev. 3.24 and IOMMU spec Rev. 2.6), it appeared that the data structures for IOMMU changed significantly. Guest Virtual APIC Table Root Pointer moved from the DTE to the IRTE, and is now strictly defined as a pointer to the virtual APIC backing page.

A useful mechanism indeed, and it's a pity it's not supported in the hardware yet.

vsinitsyn
  • 113
  • 7
  • At least in family 15h Models 60h-6Fh AVIC is supported. See http://support.amd.com/TechDocs/50742_15h_Models_60h-6Fh_BKDG.pdf – Volodymyr Buell Aug 11 '15 at 14:10
  • Good to know, thanks. Did you have a chance to try it yourself? – vsinitsyn Aug 13 '15 at 05:23
  • I tried it on my Ryzen 7, and `modprobe kvm_amd avic=1` worked. I checked `cat /sys/modules/kvm_amd/parameters/avic` and it says `1`. Tested a VM, worked fine. By default that was `0`. – doug65536 Dec 11 '17 at 04:52
2

AVIC was announced with a bit of fanfare in 2012 (see http://www.slideshare.net/xen_com_mgr/introduction-of-amd-virtual-interrupt-controller and http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-interrupt-virt-kvm-roedel.pdf) but has yet to materialize in actual hardware.

Emulating x2APIC for older hardware provides a reasonable performance boost. Qemu emulates x2APIC even for Opteron gen 1 now: http://lists.nongnu.org/archive/html/qemu-devel/2014-01/msg02441.html AVIC, as designed originally, doesn't provide x2APIC support so that might be another reason why it never took off. (See slides 2-4 in the 2nd link.)

Simultaneously, in 2012, Intel announced its own experimental "APICv" although it lacked a catchy name at its introduction http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-intel-vt-feat-nakajima.pdf. But Intel's tech appears real with benchmarks appearing in late 2013: https://software.intel.com/en-us/blogs/2013/12/17/apic-virtualization-performance-testing-and-iozone Intel's APICv is implemented at microcode level in the Ivy Bridge EP series (http://ark.intel.com/products/codename/68926/Ivy-Bridge-EP#@All), which are sold under the brand names Xeon E5-26xx v2 (introduced in late 2013) and Xeon E5-46xx v2 (introduced in early 2014).

Fizz
  • 4,782
  • 1
  • 24
  • 51