Lecture 7: Normalizing Flows 2 (KAIST CS479, Fall 2026)
Overview
Minhyuk Sung develops normalizing flows as invertible, differentiable mappings from a D-dimensional Gaussian latent space to data, allowing exact likelihoods through the change-of-variables formula. He explains how coupling layers use triangular Jacobians for tractable determinants, then connects affine autoregressive flows to autoregressive density models and shows how stacking or reordering layers expands the distributions they can represent.
Key takeaways
- A normalizing flow computes exact data likelihood by mapping data back to a known latent distribution and correcting its density with a Jacobian determinant.
- Composing flow layers adds log-determinants, so a deep mapping can remain tractable when each individual layer has an efficient inverse and determinant.
- Coupling layers obtain tractable determinants by transforming one coordinate block conditioned on another unchanged block, producing a block-triangular Jacobian.
- Affine coupling layers use exponential scales and learned shifts; their log-determinants reduce to sums of predicted log-scales.
- A single affine coupling layer leaves Gaussian, single-peaked conditional slices, but alternating blocks or stacking and permuting layers can represent more complex and multimodal distributions.
- Affine autoregressive flows correspond to autoregressive models with Gaussian conditionals: shifts act like conditional means and scales act like conditional standard deviations.
Chapters
- Contrasts VAEs, which use stochastic latent-to-data mappings and optimize an evidence lower bound, with flows, which map latent code z deterministically to data x.
- Requires a differentiable, invertible mapping between spaces of the same dimension so the data density can be computed exactly.
- In one dimension, a nonzero derivative gives a locally invertible mapping; the change-of-variables formula adjusts latent density by the transformation's derivative.
- For a sequence of transformations from h₀ to h₁ through hₙ, compute the final log density using the base density and each layer's Jacobian determinant.
- Products of per-layer determinant factors become sums of log-determinants, making deep compositions manageable.
- Layer composition increases transformation complexity while retaining tractable likelihood computation if every layer has an easy inverse and determinant.
- Stacking affine functions still produces an affine function, so affine-only layers do not increase the mapping's expressiveness.
- Elementwise nonlinear transformations also limit expressiveness because they do not mix coordinates.
- Triangular Jacobians offer a way to mix coordinates while preserving an easy determinant: the determinant is the product of diagonal entries.
- Split the input into blocks zA and zB, transform zA using both blocks, and leave zB unchanged.
- The unchanged block creates a block-triangular Jacobian, so the full determinant reduces to the determinant of the transformed block's derivative with respect to zA.
- The inverse keeps the observed xB as zB and applies the inverse transformation to recover zA.
- Alternating which block is transformed lets successive layers update all coordinates while retaining tractable inverses and determinants.
- A coupling transformation can use a neural network conditioned on zB to shift zA, while keeping zB fixed.
- For this additive coupling, the Jacobian determinant is 1, so the transformation preserves local volume and adds no log-determinant correction.
- The transformation can still be nonlinear and bend a 2D Gaussian into a complex shape, even though one coordinate remains unchanged.
- Shuffling or swapping coordinate roles across layers helps remove the fixed-coordinate limitation.
- An affine coupling layer predicts scale and shift from the unchanged block, then transforms the other block with both parameters.
- Using an exponential scale ensures positive scaling, and its log-determinant is the sum of the predicted log-scales.
- The inverse is computed by subtracting the shift and dividing by the scale, using the unchanged observed block as the conditioner input.
- Training adjusts the conditioner networks to maximize the exact log-likelihood of observed data.
- When the conditioning coordinate is fixed, an affine coupling layer maps the other coordinate affinely from its Gaussian latent value.
- Each such conditional slice therefore remains Gaussian and single-peaked, even when the overall joint distribution can look more complex.
- A target with multimodal conditional slices cannot be represented by one affine coupling layer, motivating compositions with changed coordinate roles.
- Alternate which coordinate block is held fixed and which is transformed to expose previously unchanged dimensions to nonlinear updates.
- Each layer keeps its own tractable inverse and Jacobian determinant; determinants combine across layers even if the full composed Jacobian is not triangular.
- Splitting a vector need not use equal-sized halves, allowing different block partitions to shape the transformation.
- Replace two-block coupling with coordinate-wise transformations conditioned on a prefix of earlier coordinates.
- Generation proceeds sequentially, transforming z₁ to x₁, then z₂ to x₂, and continuing through the vector.
- With the full observation x available, inversion can be parallel because each coordinate's conditioner inputs are known.
- The autoregressive Jacobian is triangular, so its log-determinant is the sum of the per-coordinate log-scales.
- A masked autoregressive network can produce scale and shift parameters for every coordinate in one forward pass during training.
- In a Gaussian autoregressive density model, each conditional outputs a mean and standard deviation; these correspond to the shift and scale in an affine autoregressive flow.
- Thus, an affine autoregressive flow is closely connected to an autoregressive model with Gaussian conditionals, while also supporting invertible transformations.
- A single affine autoregressive flow retains Gaussian conditional slices and therefore cannot capture every multimodal conditional distribution.
- Stacking flow layers and permuting coordinate order changes which variables serve as the conditioning prefix.
- The resulting composition can represent more complex conditional shapes, including multimodality that one layer cannot express.
- The central design goal is to combine expressive transformations with exact inverses and tractable Jacobian determinants.
- Coupling flows and autoregressive flows achieve this through triangular Jacobians, while stacking and reordering layers improves representational capacity.
- The next KAIST CS479 lecture will recap normalizing flows and introduce generative adversarial networks.
Transcript excerpt
0:01 Okay, so let's continue discussing the normalizing flows. So, we started to discuss normalizing flows in the week before the the previous week. So, if I just briefly recap like what we discussed last time, so VAE was kind of the first model that we discussed as kind of the some latent developed model in a way that we first basically sample from the some simple distribution typically some Gaussian distribution in a kind of like latent space and then given that kind of sample we are mapping that into the real data point. So, that was kind of the typical way that so so many other kind of the models that we are also going to discuss would be following this kind of the concept. In terms of first defining some kind of
0:48 latent space and mapping the latent sample into the data point. So, the question is that how we are going to map this point, the point Z, into the real data point. So, VAE was kind of some sort of the stochastic model in terms of that given Z we are sampling the X, right? So, we are basically learning those kind of some conditional distribution. And the kind of the downside of the VAE was that we cannot directly compute the evidence. So, which means that we cannot simply basically perform the maximum likelihood estimation with that. So, in terms of that we basically define the elbow, the lower bound of the evidence in terms of maximizing the elbow, right? And we will actually get back to those kind of the concept of the elbow next time. But the here the basic idea of the
1:36 normalizing flow is that what if we can directly map this sample Z into the one of the real data point of
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Minhyuk Sung.