Save this video free

EfficientML.ai Lecture 14 - LLM Post-Training (MIT 6.5940 Fall 2026)

MIT HAN Lab · 1:16:08 · Watch on YouTube

Part of Massachusetts Institute of Technology 6.5940 EfficientML.ai. See all 19 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Post-training turns a pretrained language model into a useful, aligned assistant through supervised fine-tuning (SFT), human preference optimization, and reinforcement learning. The lecture explains RLHF, PPO, DPO, and GRPO; parameter-efficient methods including LoRA, QLoRA, and BitDelta; and multimodal training with VILA and VILA-U, then connects prompt engineering and retrieval-augmented generation (RAG) to practical model use.

Key takeaways

Chapters

0:00 Post-Training Turns a Base Model into an Aligned Assistant
5:18 SFT, RLHF, PPO, and Direct Preference Optimization
12:03 Reasoning Models Benefit from More Inference-Time Compute
15:35 GRPO Uses Grouped Responses and Reward-Based Advantages
20:33 Speculative Decoding Reduces the GRPO Rollout Bottleneck
26:00 BitFit, TinyTL, and Adapters Reduce Fine-Tuning Costs
32:04 LoRA Adds Mergeable Low-Rank Updates to Frozen Weights
38:31 QLoRA and BitDelta Compress Fine-Tuning and Model Variants
44:19 VILA Connects Vision Encoders to Language Models
49:09 VILA Demonstrates Visual In-Context Learning and Reasoning
53:25 High-Resolution Vision and VILA-U's Unified Image Model
59:32 Few-Shot and Chain-of-Thought Prompting Guide Model Behavior
1:04:14 Safety Prompts and Detailed Diffusion Instructions
1:09:27 RAG Grounds Answers in Retrieved External Documents
1:11:52 Lecture Recap and Final Project Requirements

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:01 Okay. Uh good afternoon, everyone. Uh let's get started for today's lecture. So, today we are going to talk about uh lecture 14 on LM post-training. So, previously we already discussed how large language models are built and uh you know, deployed with some uh quantization and compression techniques. So, in this lecture we will focus on techniques makes uh uh pre-trained language model more uh powerful, useful, and better aligned to humans' preference and uh requirements. So, we will cover uh several domains such as sup- uh supervised fine-tuning, so SFT. Uh we will also talk about uh reinforcement learning and some uh parameter-efficient fine-tuning methods.

0:48 We will also talk about some uh probably uh very uh lightweight method to make large language model works better such as prompt engineering and uh retrieval arguments generation. So, let's first uh take a look at where is the post-training in the overall LM training process. So, here we have a life cycle of the large language model and we begin by uh collecting some pre-training data, very large pre-training corpus, and chance a base model using uh substantial compute. And later uh pre- a post-training happen- happens afterward with a much uh smaller, more targeted data set. Uh its purpose is to teach the model how to follow instructions, behave safely, and performs the task we care about.

1:36 And finally, uh we have some deployment organizations uh to improve the latency and memory efficiency uh as we have always been uh talking about. And pre-training creates a general capability and the world knowledge inside the large language models. Post-training turns that capability into a uh useful product or assistant at into a many applications. So, nowadays uh post-training uh typically requires much smaller data set compared to pre-training. It's already It's already become more and more

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT HAN Lab.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.