cs6465 Advanced OS - Lecture 05 - Intel VMX (part 3)
Watch on YouTube →
Overview
Anton Burtsev explains how Intel VMX reduces virtualization overhead through Extended Page Tables (EPT) and virtual APIC support, with EPT replacing costly shadow-page-table maintenance and posted interrupts enabling device interrupts to reach a guest without a VM exit. The lecture also covers EPT performance and protection features, TLB identifiers, migration logging, and the VT-d interrupt-remapping path used for efficient and secure device passthrough.
Key takeaways
- EPT removes the need for the hypervisor to synchronize shadow page tables on ordinary guest page-table changes by translating guest physical addresses to host physical addresses in hardware.
- Nested translation can be costly because EPT walks are needed to access guest page-table entries; 2 MB and 1 GB EPT pages reduce walk depth when memory can be mapped contiguously.
- PCID tags guest-process translations, VPID tags VM translations, and EPT context tags distinguish translations when VMFUNC switches between EPTs.
- PML records up to 512 dirty guest-page entries per buffer, helping live migration avoid scanning the full page-table tree on each transfer round.
- VT-d interrupt remapping validates the source device and permitted interrupt vector, while posted interrupts update guest virtual APIC state so passthrough interrupts can arrive without a VM exit.
Chapters
- The lecture focuses on Intel VMX chapters covering Extended Page Tables (EPT) and Advanced Programmable Interrupt Controller (APIC) virtualization.
- Both mechanisms aim to reduce VM exits, which is especially important for memory-intensive and I/O-intensive workloads.
- Virtual APIC support makes low-overhead interrupt delivery and device passthrough possible.
- Before EPT, a hypervisor maintained shadow page tables mapping guest virtual addresses directly to host physical addresses.
- The hypervisor had to keep shadow mappings synchronized with guest page-table changes, often by trapping writes or handling page faults.
- EPT adds a second translation: guest page tables map guest virtual to guest physical addresses, then EPT maps guest physical to host physical addresses.
- A four-level EPT walk can add memory accesses while the processor also reads guest page-table entries, making nested translation expensive.
- Using 2 MB or 1 GB EPT pages shortens the EPT walk when the guest memory layout permits larger mappings.
- Large pages trade finer-grained allocation for translation efficiency; systems commonly use 2 MB pages, while 1 GB pages can reduce walk depth further.
- The TLB caches virtual-to-physical translations so the processor can avoid repeatedly walking page tables.
- A guest-managed PCID tags translations for guest processes, while the hypervisor-managed VPID distinguishes virtual machines.
- Using both identifiers lets guest process switches and VM switches retain valid TLB entries instead of flushing them every time.
- Intel also tags translations with an EPT context because VMFUNC can switch between EPT contexts without changing the VMCS or guest PCID.
- Without the EPT context tag, translations from one EPT could be incorrectly reused after switching to another.
- VMFUNC can change the guest's memory view quickly, but managing EPT contexts and stale translations makes practical uses more complex.
- Hypervisor-managed Linear Address Translation (HLAT) lets the hypervisor control translation for selected guest linear-address ranges without trapping every page-table write.
- An HLAT bit in a guest page-table entry causes the processor to restart translation through a hypervisor-controlled HLAT table.
- HLAT can protect a fine-grained region from guest remapping, addressing attacks that alter page-table mappings rather than modifying protected code pages.
- EPT permissions are independent of guest page-table permissions, so the hypervisor can make a guest-writable page read-only at the host-controlled EPT level.
- An EPT violation occurs when an access conflicts with EPT permissions or encounters an absent mapping; an EPT misconfiguration indicates an invalid EPT entry.
- A hypervisor can use an absent EPT mapping for demand paging: handle the fault, install a backing-page mapping, then resume the guest.
- Intel can deliver some EPT violations to a guest's virtualization-exception handler instead of always causing a VM exit.
- A security monitor can use the exception to inspect access to protected data, such as a key, and decide whether to allow or deny it.
- VMFUNC can switch EPT contexts as part of such a policy, though changing the guest's memory view complicates security guarantees.
- Sub-page write permissions refine access control within a 4 KB page, using 128-byte granularity and a bitmap-like structure.
- The mechanism can distinguish fields in a data structure, allowing selected regions to be writable while others remain read-only.
- The extra lookup is distinct from the repeated nested page-table walks that make small EPT pages costly, but the lecture notes that real-world performance evidence is limited.
- Live migration copies guest memory while the VM continues running, then recopies pages dirtied during earlier transfer rounds.
- Intel Page Modification Logging (PML) records guest physical pages when their EPT dirty state is first set, avoiding a full page-table scan.
- The PML buffer holds 512 entries; repeated copy rounds shrink the remaining dirty set before the VM is briefly paused for final transfer.
- A PCI device can signal an interrupt using a Message Signaled Interrupt (MSI), which is routed through interrupt-control hardware.
- The IOAPIC routes interrupts to cores, while each core's local APIC tracks pending interrupts and priorities.
- The local APIC delivers an interrupt when the processor can accept it, and the device driver services the device and acknowledges completion.
- APIC virtualization gives a guest virtual interrupt-controller state so pending guest interrupts can be tracked without always involving the hypervisor.
- The Interrupt Request Register (IRR) records pending vectors; delivery moves a vector into the In-Service Register (ISR).
- After the handler completes, the guest issues End of Interrupt (EOI), allowing the APIC to deliver another eligible interrupt.
- With external-interrupt exiting disabled, interrupts can be delivered directly to the guest, a configuration that is uncommon when the hypervisor needs control.
- With exiting enabled, an interrupt causes a VM exit, after which the hypervisor can acknowledge it and inject it into the guest.
- Depending on acknowledgment settings, the guest may handle delivery after re-enabling interrupts, but both paths still incur an exit that APIC virtualization seeks to avoid.
- Without posted interrupts, a device interrupt targeting a guest exits to the hypervisor, which then injects the interrupt into the guest.
- Intel VT-d interrupt remapping checks which device may send a given interrupt and identifies its destination VM.
- For passthrough, the hypervisor configures the device-to-guest route and a posted-interrupt descriptor for the guest's virtual APIC.
- A device interrupt refers to an entry in the interrupt remapping table, where hardware validates the device identity and permitted vector.
- For a posted interrupt, remapping hardware updates the guest's virtual pending-interrupt state through the posted-interrupt descriptor.
- The guest's virtual APIC then delivers the vector through the guest IDT when delivery is permitted, avoiding a VM exit.
- A passed-through network interface can send interrupts directly to its assigned guest using VT-d interrupt remapping and posted-interrupt state.
- The essential combination is hardware validation of device, vector, and destination plus virtual APIC delivery without a hypervisor round trip.
- The next lecture will finish the discussion of TLB shootdowns and move to techniques for implementing fast virtualized device drivers.
Transcript excerpt
0:39 Go. Okay. So, I wanted to finish uh the VN invite all the mechanisms essentially. is the last uh lecture on it and then we'll just uh like after that you'll you'll understand everything I mean like at least you will read about everything right uh and uh and after that we will just move in move on to like uh implementing like discussing how to implement fast uh uh virtualization of device drivers different techniques but uh this lecture is a little I mean it's very relevant
1:27 for two reasons Because specifically this lecture discusses two things. Uh it talks about virtualization of uh physical memory with extended page tables. But even more important it talks about virtualization of advanced programmable interrupt controller which is like super critical for delivering interrupts in an efficient manner and for implementing device pass through. And uh both of these techniques I mean like EPD obviously is used everywhere but uh uh virtualization of advanced programmable interrupt controller like this is what enables like like modern low overhead virtualization of IO intensive workloads and this this was absent for for a long time for over 10 years like if if the first if the first
2:16 versions of uh Intel BMX were released like around 200 5 maybe 2006. Uh essentially virtualized Apex showed up only like a handful of years ago, maybe like five years ago. So it's been a while, but I'll explain it. So, and I I assume you guys all can hear me, right? So, sorry. Is it this one? Thank you. I just wanted to remove this panel. Okay, cool. So, where am I now? My screen go my slide deck. That's the question was presenting in PowerPoint. Where did
3:06 it go? Here. Okay. Cool. Okay. So, sure. Let's talk about these two mechanisms
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Anton Burtsev.