cs6465 Advanced OS - Lecture 08 - HW2 Discussion (xv6 on top of xv6)
Watch on YouTube →
Overview
Anton Burtsev develops a practical design for running xv6 as a VMX non-root guest on an xv6-based hypervisor, starting from the kernel’s 32-bit entry point and loading ELF segments through an EPT-backed guest memory interface. He works through the minimum filesystem changes, device emulation, interrupt handling, and single-CPU assumptions needed to reach an interactive guest, while explaining how to keep most device emulation in user space.
Key takeaways
- The xv6 kernel can start directly in a VMX guest at its 32-bit ELF entry point because xv6’s own bootloader already performs the 16-bit-to-32-bit transition before jumping to that entry.
- Classic xv6 file limits are far too small for a kernel image: with 512-byte blocks and 128 pointers per indirect block, capacity is only about 70 KB; a multi-level radix tree can raise the limit to roughly 8 MB or more.
- A guest-memory loading interface should translate guest physical addresses through the EPT and copy data page by page, allowing ELF segments and device buffers to reside in noncontiguous host memory.
- Starting with one guest CPU sharply reduces hypervisor scope by eliminating multiprocessor startup, interprocessor interrupts, and cross-core TLB coordination, though xv6 still needs valid MP configuration data.
- Device emulation can remain outside the hypervisor core: VMX reports I/O exits and register state, while a user-space VM runner emulates UART and IDE operations and updates the guest state.
- Guest and host interrupt responsibilities differ: the host timer supports host scheduling and input polling, while injected guest timer vector 32 lets xv6 perform its own scheduling.
Chapters
- Anton Burtsev frames Homework 2 as adapting the Homework 1 hypervisor to run an xv6 guest.
- The exercise is intended to build a reusable mental model for designing a hypervisor, with Linux as a possible later extension.
- The discussion focuses on a plan for booting the guest rather than simply following implementation instructions.
- The xv6 BIOS boot path starts with a 16-bit boot block, then switches to 32-bit mode before loading the kernel.
- The bootloader loads the ELF kernel and jumps to the entry point in entry.s, which is already prepared for 32-bit execution.
- A VMX guest can therefore start directly at the kernel entry point, avoiding emulation of the bootloader’s 16-bit transition.
- xv6 stores file metadata in inodes, whose block pointers include direct entries and a final pointer to an indirect block.
- With 512-byte disk blocks and 4-byte block addresses, an indirect block holds 128 pointers.
- The classic xv6 layout supports only about 70 KB per file, too little for a reasonably sized kernel image.
- A linked list of file blocks grows files but requires sequential traversal, making access scale linearly with file length.
- A radix tree uses block-number bits to select entries across levels, similar to how a page table indexes addresses.
- With 128-way levels and 512-byte blocks, two levels can address roughly 8 MB; adding another level expands capacity further.
- The hypervisor reads the xv6 ELF header and loadable segments from the host filesystem, rather than copying the whole ELF file as one contiguous image.
- The guest kernel is linked to load around 1 MB, so the loader must honor each segment’s guest physical address and size.
- A VM memory-load system call can walk the EPT page by page and copy host buffers into the corresponding guest physical pages.
- x86 keyboard input and serial input use different device paths; the guest needs a way to receive keystrokes from the host terminal.
- The proposed minimal implementation uses a serial interface rather than emulating both keyboard hardware and UART input.
- The guest’s serial input path ultimately requires injecting an interrupt after the host-side VM runner detects available input.
- xv6 console output can use both a serial UART and a text-video buffer associated with CGA or VGA.
- The initial hypervisor can skip VGA memory emulation and support serial output through port I/O.
- When guest code writes a character to the UART port, the VM runner can print that character to the host terminal.
- The IOAPIC routes device interrupts to local APICs, while LAPICs provide per-core interrupt handling and interprocessor interrupts.
- A single-CPU guest avoids emulating CPU wakeups, interprocessor interrupts, and cross-core TLB shootdowns.
- The proposed first implementation can inject guest interrupts directly, while providing simple CPU identification and tolerating limited interrupt-controller behavior.
- xv6’s multiprocessor initialization searches for BIOS-provided MP configuration information describing CPUs and interrupt-controller locations.
- The guest does not run the normal BIOS path, so the hypervisor must place an MP floating pointer structure and MP configuration table in guest memory.
- The tables can describe a one-CPU system, allowing xv6 to proceed without full LAPIC and IOAPIC emulation.
- VMX I/O exits identify the operation and provide details such as the I/O port, direction, and instruction length.
- The kernel can return device-emulation exits to a user-space VM runner, which handles them with a switch statement or per-port handlers.
- The VM interface can expose guest registers and memory operations, keeping device-specific logic outside the hypervisor core.
- A guest UART write executes an x86 OUT instruction, which can be trapped through the VMX I/O bitmap.
- The emulator reads the port and data value from the exit information and guest register state, then writes the character to host output.
- After emulating the instruction, it must advance the guest instruction pointer by the VMX-reported instruction length to prevent the same OUT from repeating.
- The REP OUTS instruction transfers multiple values from guest memory to an I/O port, using a count in RCX and a buffer address in a register.
- The VM runner can emulate the whole string operation rather than handling each repeated transfer as a separate exit.
- Guest-memory access must work across page boundaries; the EPT-backed memory interface can copy the buffer page by page for IDE emulation.
- The guest is configured so selected interrupts cause VM exits, after which the host handles the hardware interrupt and decides whether to inject an event into the guest.
- Pending events must be tracked and injected one at a time so that overlapping events, including a guest syscall and a host interrupt, are not lost.
- The proposed minimal guest interrupt set includes serial input, IDE completion, and timer interrupts; syscall handling needs care if it overlaps an external event.
- A VM runner executing the guest cannot simultaneously block while reading standard input, so it needs periodic opportunities to check for keystrokes.
- A nonblocking host read can detect input without sleeping when the terminal has no data.
- When a character is available, the runner uses VMX interrupt injection to deliver the corresponding serial event to the guest.
- For an IDE read, the VM runner can perform the disk operation synchronously instead of building an asynchronous device queue.
- After the read finishes and its data has been copied into guest memory, the runner injects the completion interrupt.
- This avoids polling for disk completion, unlike serial input, which requires checking the host terminal for new keystrokes.
- Host timer events let the hypervisor regain control, give other host processes time to run, and later resume the guest.
- When a timer tick has elapsed, the runner injects guest interrupt vector 32 so xv6 can run its own scheduling logic.
- Timer exits also provide regular opportunities to poll standard input, preventing a guest loop from keeping the VM runner from noticing keyboard activity.
- Burtsev says Homework 2 will follow the format of the previous assignment and asks students to study the accompanying implementation write-up.
- The combined Homework 1 and Homework 2 exercises are intended to establish the foundation for later virtualization work.
- The closing discussion clarifies that host round-robin scheduling and guest xv6 scheduling are separate: the host schedules processes, while injected guest timer interrupts let xv6 schedule internally.
Transcript excerpt
0:53 Yes, so welcome back everyone. Um So, what I wanted to do today is virtually talk through the homework two. Uh and again, the reason is that uh I can publish it. I can do everything. But I'm still not sure whether that you really learn from it. Just because, you know, you can either chat GPT or maybe like scroll through the blog posts and stuff. So, my goal today is to essentially, assuming that you worked on a plan for how to build it, I'll brainstorm with you for how can we build a hypervisor, which which actually or how can we change homework one rather uh in such a manner that we can actually boot X 386 on the lower axis. All right?
1:40 Make sense? By the way, any any questions about uh about the homework homework one? >> I didn't really figure out, but I was getting an error about deep stack bound in BMA. >> Uh-huh, but I fixed it and replied to your a I think it's fixed. >> I know that it was like Yeah, it was uh just two specific grab for specific clients. That was fixed. I think you getting concrete right now, but you fixed it. One way or another, you already fixed it. Right? Otherwise, no questions. Okay, guys. So, then like let's let's jump on it and tell me like what is that we need to do uh to to boot X with 6 and double of X with 6.
2:29 We don't have to rush. We have the boat. The reason The reason I'm doing it because this is essentially the essence of what it that I wanted to teach you. Like if someone approaches you tomorrow like or in half a year and ask you, "Okay, build me a hypervisor."
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Anton Burtsev.