Save this video free

Lecture 9: Energy-Based Models 1 (KAIST CS479, Fall 2026)

Minhyuk Sung · 1:02:03 · Watch on YouTube

Part of KAIST AIC313/CS378. See all 12 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Minhyuk Sung develops energy-based models (EBMs), where a neural network assigns each data point a scalar energy and defines its probability density through a normalized exponential of negative energy. He derives the maximum-likelihood gradient as a competition between lowering energy on training examples and raising it on samples from the current model, then introduces Markov chain Monte Carlo and Langevin dynamics for sampling without computing the partition function.

Key takeaways

Chapters

0:00 GAN Recap: Discriminator Signals and Generator Imbalance
4:12 Energy-Based Models: Turning Neural Network Outputs into Densities
11:15 Partition-Function Ratios and Composing Energy Functions
19:00 Maximum Likelihood Produces Data and Model Expectations
27:22 Monte Carlo Gradient Estimates and a Gaussian Check
35:20 Why EBMs Need a Sampling Procedure
38:29 Markov Chains and Stationary Target Distributions
43:51 Langevin Dynamics: Energy Gradients Plus Gaussian Noise
48:52 Burn-In and Practical Convergence in Langevin Sampling
55:07 Sampling Without the Partition Function
58:32 Score Functions: The Gradient of Log Probability
1:01:10 Langevin Noise Versus Gradient Descent

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:02 Okay. So on last week uh we discussed the another kind of model which was the generative adversarial neuron net. Uh we chose the case that now we are training not a single neuronet but actually the two neuron nets and also making them like compete each other in terms of that we are having the uh the generator as usual and then we are also having the discriminator which is basically you know checking whether it's the output is kind of like real or the fake. uh so by jointly training this kind of the binary declassifier which is giving some kind of some learning signal uh to the other kind of the generator right so this was kind of the case that we are basically training our kind of model without explicitly basically evaluating the probability density for any kind of the data points. So that is kind of the the

0:50 case of the uh the generator. So we also discussed the kind of the the minmax to objective function to train those kind of the two neural networks and we discussed kind of the meaning of those kind of the objective function right. So when you basically see the minity function when you fix the the generator and basically assume some kind of equality prior for the real and the fake labels we could see that in the optimal case our the discriminator should return this specificity ratio right this kind of quantity which is having this kind of meaning right so when we have the optimal discriminator if the output is greater than 0.5 that's the case that our generator is the 0.5 at the certainly data point X here. Uh this is

1:38 kind of the region which is basically the hourly generator is kind like under

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Minhyuk Sung.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.