Lecture 6: Normalizing Flows 1 (KAIST CS479, Fall 2026)
Overview
Minhyuk Sung introduces normalizing flows as deterministic, invertible transformations from a simple latent prior to data, enabling exact likelihood evaluation through the change-of-variables formula. He derives the one- and multidimensional density corrections, explains the invertibility and Jacobian-computation challenges, and compares affine and elementwise monotonic transformations with the limitations each imposes.
Key takeaways
- A normalizing flow computes exact data likelihood by mapping x back to z, evaluating the prior density, and multiplying by the absolute determinant of the inverse map's Jacobian.
- Invertibility is essential: in one dimension the transformation must be strictly monotonic, and in multiple dimensions its Jacobian determinant must remain nonzero.
- The Jacobian determinant is a local volume-change factor; expansion lowers density, while contraction raises it, preserving probability under the transformation.
- VAE loss can decrease during posterior collapse because the KL term approaches zero even while the decoder ignores latent codes and reconstruction quality suffers.
- Affine flows are easy to invert and evaluate but stacking them remains affine, limiting their ability to model multimodal data; elementwise monotonic flows add nonlinear flexibility but still lack coordinate mixing.
Chapters
- A VAE can use a latent space with a different, often lower, dimension than the data space.
- Exact evidence evaluation requires marginalizing over latent codes, an integral that is generally intractable.
- Training maximizes the ELBO over data samples, typically estimated with minibatches.
- The expected ELBO can be interpreted through a KL divergence between joint distributions over data x and latent code z.
- One joint distribution samples z from the prior and x from the decoder; another starts with observed x and samples z from the encoder.
- The objective balances matching the data distribution against making the encoder approximate the decoder-defined posterior.
- A diagonal Gaussian encoder may poorly approximate a decoder posterior with multimodal or otherwise complex structure.
- Limited encoder expressiveness can leave the posterior-matching KL term high even when the decoder models the data well.
- Because the ELBO trades off data fit and posterior fit, optimizing one term can come at the expense of the other.
- Posterior collapse occurs when approximate posteriors approach the prior, such as a standard Gaussian, and z carries little information about x.
- A decoder can then ignore z, harming reconstruction even as the KL-to-prior term falls toward zero.
- A decreasing total loss alone does not prove VAE training is healthy; reconstruction and latent-use behavior also need inspection.
- Autoregressive generation proceeds sequentially, which can require one generation step per token or pixel.
- A VAE samples a latent code and decodes from it, allowing parallel generation rather than a long sequence of conditional steps.
- VAE latent representations support operations such as interpolation, but approximate likelihood training and blurry outputs can be drawbacks.
- Unlike a VAE decoder that samples from a learned likelihood, a normalizing flow maps latent z to data x with a deterministic function f.
- The lecture assumes equal latent and data dimensions for the basic construction.
- The goal is to transform a simple prior, typically a standard Gaussian, into a distribution aligned with the observed data.
- Stretching a uniform interval from [0, 1] to [0, 2] halves its probability density so the total probability remains one.
- A first-order Taylor approximation describes how a small interval changes under a differentiable transformation.
- In one dimension, density must be corrected by the local stretching factor, using the absolute derivative of the inverse map.
- A one-dimensional flow needs a one-to-one transformation, such as a strictly increasing or strictly decreasing function, so an inverse exists.
- For observed x, compute z = g(x), evaluate the prior density at z, and multiply by the absolute derivative of g.
- Learning a neural transformation that stays invertible is a central modeling challenge, not an automatic property of an arbitrary network.
- In multiple dimensions, the Jacobian matrix generalizes the derivative and its determinant measures local volume change.
- The density is p_X(x) = p_Z(g(x)) times |det J_g(x)|, where g is the inverse transformation.
- The Jacobian determinant must be nonzero for the transformation to be locally invertible and the density correction to be defined.
- For the lecture's two-coordinate example, the observed point x = (1, 1) maps back to latent code z = (1, 0).
- Evaluating the standard Gaussian prior at that code supplies the latent density.
- The Jacobian determinant is 2 at z = (1, 0), indicating a twofold area expansion and a corresponding density correction.
- An affine flow x = Az + b is invertible when A is nonsingular; solving a linear system for z can be more numerically stable than explicitly computing A inverse.
- Diagonal or triangular matrices can help guarantee invertibility, while diagonal-only transformations limit coordinate mixing.
- Stacking affine transformations still produces an affine map, so it cannot represent complex multimodal distributions from a Gaussian prior.
- Elementwise monotonic nonlinear maps, including spline-based flows with adjustable knots, add flexibility but do not mix coordinates; richer flow designs must address both expressiveness and efficient inversion.
Transcript excerpt
0:01 So yeah let's continue discussing uh the so we discussed almost like everything about the last time like the only basic idea is that we are having this kind of the structure uh in terms of that we are defining some kind of the prior distribution the latent distribution and taking some kind of latent code first sampling from here and just mapping this to the data distribution uh by like the sampling from the another kind of the likelihood distribution that we are learning using the decoder, right? So that's kind of the basic thing. Uh but by the way, we didn't explicitly talk about this, but now when you think about kind of the dimension of the data distribution and also the latent distribution, should they have the same dimension uh for both of them? Uh not necessary, right? I mean it can
0:53 have some kind of different dimension for the data and the lat here. So as we also do some kind of the uh some bottleneck layer here in terms of having the the lower dimensional the space for the latent in the autoenccoder we can also set some kind of the lower dimensional space uh in the latent space especially like when you assume that the data distribution will have some kind of the some simpler the structure like know having some kind of the even in the higher dimensional space if the data will be placed in some kind of the lower dimension the subspace then we can easily basically reduce the dimension of the lat code here. So that's kind of thing. But actually you you need to keep this structure in mind because this structure is not only for the bae but also for the many other things also for the normalizing flow and g diffusion
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Minhyuk Sung.