Lecture 12: Denoising Diffusion Probabilistic Models 1 (KAIST CS479, Fall 2026)

Overview

Minhyuk Sung introduces denoising diffusion probabilistic models (DDPMs) as a route to high-quality, diverse generation, contrasting their slower iterative sampling with the speed and limitations of VAEs and GANs. He derives DDPMs from the perspective of a Markovian hierarchical VAE: the forward Gaussian process is predefined to turn data into noise, while a learned reverse process reconstructs data, with training details deferred to the next lecture.

Key takeaways

DDPMs trade sampling speed for quality and diversity: generation applies a neural network repeatedly across denoising steps, and reducing those steps is a central acceleration challenge.
A DDPM can be viewed as a Markovian hierarchical VAE whose latent states match the data dimensionality, with a fixed forward encoder and a learned reverse decoder.
A non-decreasing beta_t schedule, with alpha_t = 1 - beta_t, controls the forward corruption; choosing alpha_bar_T near zero makes the terminal state approximately standard Gaussian.
The cumulative forward process has a closed-form Gaussian distribution for X_t given X0, so training can draw noisy examples at arbitrary time steps without simulating all prior transitions.
Although X_(t-1) given Xt alone may not be analytically tractable, conditioning on both X0 and Xt yields a Gaussian reverse posterior with computable variance and mean.
The reverse network learns to predict the posterior mean from Xt and t without access to X0, enabling generation from Gaussian noise through repeated reverse transitions.

Chapters

0:00 Why Diffusion Supplanted GANs in Image Generation

VAEs and normalizing flows offer fast sampling and diversity but often produce lower-quality outputs; GANs can produce sharp outputs but face mode collapse and diversity challenges.
StyleGAN demonstrated strong face generation around 2019, but its capabilities were more limited than later systems for arbitrary image prompts.
From 2022 to 2023, diffusion image systems improved rapidly, moving from incoherent images toward near-photorealistic prompt-based results.

5:30 Diffusion and Flow Models Across AI Applications

Diffusion and flow-based models are used for image and video generation, and diffusion architectures also appear in AlphaFold 3 and protein-design systems.
The lecture cites applications in material generation, climate modeling, and world models such as World Labs' 4D scene reconstruction.
In robotics, diffusion policies condition on environment information to generate actions; diffusion-based approaches are also being explored for language modeling.

11:00 The Diffusion Tradeoff and Three Ways to Interpret DDPMs

Diffusion models combine high output quality with broad diversity, but conventional generation requires many neural-network denoising steps.
The course will study ways to accelerate sampling, connect diffusion to flow models, and apply diffusion to conditioning, reward alignment, distillation, and personalization.
Sung frames diffusion through three perspectives: variational autoencoders, energy-based models and score matching, and flow models.

13:00 VAE Recap: Gaussian Latents and Posterior Limits

A VAE uses a prior, typically a standard Gaussian, a decoder to model the likelihood, and an encoder to approximate the posterior.
For continuous data, common VAE encoders and decoders predict Gaussian means and variances with neural networks.
A Gaussian approximate posterior can restrict encoder expressiveness, creating a tradeoff between posterior approximation and data fit in the VAE objective.

16:00 Sequential VAEs Build Expressiveness with Markov Latents

A hierarchical or sequential VAE replaces one latent variable with a chain, using X0 for data and X1 through XT for latent states.
Under a Markov assumption, each state depends only on its neighboring state, and the joint distribution factors into a product of transition conditionals.
Nonlinear transition means and state-dependent variances can make the resulting distribution non-Gaussian, but stacking encoder and decoder networks increases training complexity.

22:00 DDPM as a Special Markovian Hierarchical VAE

Sung presents the basic DDPM as a special case of a Markovian hierarchical VAE with a sequence of latent states.
Each latent state has the same dimensionality as the data, as in a normalizing flow, rather than compressing the data into a smaller latent representation.
Unlike a conventional VAE that learns both encoder and decoder, DDPM fixes the forward encoder transitions and learns the reverse decoding transitions.

26:00 The Forward Gaussian Process and Noise Schedule

The forward process maps X0 toward XT through Gaussian transitions controlled by a time-dependent beta_t.
The noise schedule beta_t is non-decreasing; alpha_t is defined as 1 minus beta_t and therefore decreases over time.
Possible schedules include constant, linear, quadratic, and cosine-based functions; reparameterization samples each transition using standard Gaussian noise.

33:00 Directly Sampling a Noisy State from the Original Data

Because forward transitions have linear means and Gaussian noise, multiple steps can be combined into a Gaussian conditional distribution for X_t given X0.
The cumulative coefficient alpha_bar_t is the product of the alpha values up to time t, yielding a direct sampling route without simulating every intermediate state.
This closed form lets training sample arbitrary noise levels directly from a data point, rather than running the full forward chain for each example.

42:00 How the Schedule Turns Data into Standard Gaussian Noise

The schedule is designed so alpha_bar_T approaches zero, making the conditional distribution of XT given X0 approach a standard Gaussian.
At early steps, the conditional mean stays near X0 and variance is small; over time, the mean moves toward the origin and variance grows.
At the terminal step, the original data contributes negligibly, allowing generation to begin from a standard Gaussian sample.

45:00 Deriving the Reverse Posterior with Both X0 and Xt

The reverse transition X_(t-1) given Xt alone may lack a tractable analytic form, even though each forward transition is Gaussian.
Conditioning on both X0 and Xt makes the reverse posterior tractable: Bayes' rule combines the forward likelihood, the prior transition, and the marginal distribution.
The resulting conditional is Gaussian, with a computable variance and a mean that is a linear function of X0 and Xt.

52:00 Learning the Reverse Mean to Generate Data from Noise

During training, X0 and Xt are both available, so the analytic reverse-posterior mean can be computed; at generation time, X0 is unknown.
A neural network takes Xt and time step t as inputs and learns to predict the reverse mean; the variance can be calculated from the predefined schedule.
Sampling starts from XT drawn as Gaussian noise, repeatedly predicts and samples X_(t-1), and eventually produces X0; the next lecture derives the ELBO objective and loss.

Transcript excerpt

0:01 Okay, so until the midterm the week we discuss kind of like bunch of the kind of the joint models starting from the auto list model in the VAE normalizing flow again and the energy based model. Uh, you're going to see that actually these are kind of sort of the rubbing up uh, in terms that we've been some kind of the basic the the generative models and after the midterm we're going to focus on basically discussing the kind of the most recent model which is the diffusion model and also the flow based model. So, so we're going to focus on discussing these model after the midterm. So, basically what we have seen so far is that we discuss that those models in terms of the

0:48 uh, the uh, sorry. The GAN the VAE and the normalizing flow these are kind of the good in terms of that you know, we can achieve some kind of the fast the sampling the speed and also achieving some kind of the all the diversity but we did the VAE and the normalizing flow the quality is basically not good, right? So, we have a discussed the another model which is the GAN which is kind of good in terms of achieving basically very high quality while having some kind of the difficulties in terms of like resolving the mode collapse and also achieving the high diversity. But actually if I just briefly explain like what happened around like 2019 and the 2020 uh, until this time actually for the you know, language modeling you know, people just know use

1:37 the the auto auto list model and the auto list model basically kind of the exploded with transformer around like this time and we started to see some kind of the

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Minhyuk Sung.