Save this video free

EfficientML.ai Lecture 18 - Diffusion Model (Part II) (MIT 6.5940 Fall 2026)

MIT HAN Lab · 1:06:03 · Watch on YouTube

Part of Massachusetts Institute of Technology 6.5940 EfficientML.ai. See all 19 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Diffusion inference can be accelerated along three complementary axes: sparse attention, low-bit quantization, and fewer denoising steps. The lecture covers Sparse VideoGen and its semantic-clustering extension, Radial Attention's O(L log L) attention pattern, SVDQuant and Delta-DiT for quantization, QuantVideoGen for two-bit KV-cache compression, and AnyFlow for flexible-step video diffusion distillation.

Key takeaways

Chapters

0:00 Diffusion Inference Optimization: Sparsity, Quantization, and Distillation
2:00 Video Attention's Locality Creates Opportunities for Sparsity
6:36 Sparse VideoGen Classifies Spatial and Temporal Attention Heads
9:39 SVG Reorders Tokens for Hardware-Efficient Sparse Kernels
13:27 Sparse VideoGen V2 Uses Semantic Clustering to Select Blocks
23:10 Radial Attention Allocates Compute by Spatial and Temporal Distance
28:50 Radial Attention Extends Video Length with Lower Training Cost
32:32 FP4 Makes W4A4 Diffusion Inference Attractive but Difficult
37:00 SVDQuant Separates Weight Outliers into a Low-Rank Branch
46:48 Delta-DiT Quantizes Video Activation Differences
51:49 QuantVideoGen Compresses Autoregressive Video KV Caches to Two Bits
57:40 AnyFlow Distills Video Diffusion for Flexible Sampling Steps
1:05:00 AnyFlow Results and the Three Diffusion Acceleration Axes

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:02 Uh okay let's get started for today's lecture and today we are going to talk about the uh second part of the diffusion model. So we talked about the uh basic uh definition of diffusion modeling uh in the last lecture and about about the uh how is the diffusion model the noise from the uh random noise to uh for example image or video uh set as a output. And we also talk about some uh optimization techniques such as uh the deep compression autoenccoder for uh higher compression ratio when you convert the image or video into the latent space so that you can save a lot of tokens uh during the image and video generation. And we also talk about the

0:49 uh hybrid linear attention architecture which makes uh attention operation more efficient uh on the uh multimodel generation. So today we are mainly going to talk about uh some inference optimizations uh for particularly for the diffusion modeling um we'll talk about sparity uh quantization and distillation respectively. So um first of all uh we are going to discuss some uh sparsity techniques in uh division models and the central question here is uh how to preserve the general quality while spending less computation. So you can uh easily map this kind of idea to the uh concept of

1:39 uh sparsity such as sparse attention. We have talked in previous uh lectures about uh for example long context language models and uh so observation in diffusion models are quite similar to what we met in the long context language models. We can see that uh video diffusion models can be very expensive uh as a tension layer. That's because um in video generation we have uh uh each frame is a image and a video is consist of uh a lot of

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT HAN Lab.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.