Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 3: Calculus of Variations

Overview

Daniele Gammelli introduces the Calculus of Variations as a method for finding optimal control signals in infinite-dimensional optimization problems, building upon finite-dimensional optimization concepts like necessary optimality conditions. The lecture covers the Karush-Kuhn-Tucker (KKT) conditions for inequality constraints and then transitions to the Calculus of Variations, defining functionals, norms, and differentiability to derive the fundamental theorem of calculus of variations, which leads to the Euler-Lagrange equation for optimal control.

Key takeaways

The Karush-Kuhn-Tucker (KKT) conditions provide necessary optimality conditions for problems with inequality constraints, leveraging the concept of active constraints.
Indirect methods in optimal control derive necessary optimality conditions (often differential equations) and solve them to find candidate solutions.
Calculus of Variations extends finite-dimensional optimization concepts (functionals, norms, differentiability) to infinite-dimensional problems.
The Fundamental Theorem of Calculus of Variations states that the variation of a functional must be zero at an extremum.
For a simplified optimal control problem with fixed endpoints, the variation leads to the Euler-Lagrange equation: d/dt (∂g/∂x') - ∂g/∂x = 0.

Chapters

0:00 Review of Optimal Control and Finite-Dimensional Optimization

Optimal control aims to minimize an objective function (terminal cost + running cost) for a dynamical system.
Finite-dimensional optimization reviewed: necessary optimality conditions for unconstrained and equality-constrained cases.
Focus shifts to inequality constraints and the 'easy route' via active constraints.

5:38 Inequality Constraints and Active Constraints

Inequality constraints: gj(x) <= 0.
Active constraints are those met at equality (gj(x) = 0) at a feasible point.
Inactive constraints are strictly less than 0 (gj(x) < 0).

11:53 Optimality for Inequality Constraints: The Active Set

A local minimum of the full problem is also a local minimum for the problem with only active constraints.
Intuition: inactive constraints can remain inactive in a small neighborhood around the minimum.
Active inequality constraints can be treated similarly to equality constraints for optimality conditions.

23:31 Lagrangian and Optimality Conditions for Equality Constraints

Review of the Lagrangian L(x, lambda) = f(x) + sum(lambda_i * h_i(x)).
Necessary optimality condition: gradient of the Lagrangian with respect to x is zero.
This leads to a system of n + m equations for n + m unknowns.

26:40 Extending Optimality Conditions to Inequality Constraints

Apply the same reasoning as equality constraints, incorporating active inequality constraints.
Introduce Lagrange multipliers mu_j for inequality constraints.
Optimality conditions include gradient of Lagrangian = 0, mu_j >= 0 for active constraints, and mu_j = 0 for inactive constraints.

36:52 Karush-Kuhn-Tucker (KKT) Conditions

Formal statement of necessary optimality conditions for inequality-constrained problems.
Conditions: gradient of Lagrangian = 0, mu_j >= 0, and mu_j * g_j(x) = 0 (complementary slackness).
Requires linear independence of gradients of active constraints (constraint qualification).

42:27 Example: Applying KKT Conditions

Minimize f(x, y) = x^2 + y^2 subject to g(x, y) = 2x + y - 2 <= 0.
Case 1: Constraint is active (2x + y = 2). Leads to contradiction with mu >= 0.
Case 2: Constraint is inactive (2x + y < 2). mu = 0, gradient of Lagrangian = 0 yields x=0, y=0, which is the minimum.

55:41 Roadmap: Open-Loop vs. Closed-Loop Methods

Focus for next 2-3 weeks: open-loop methods for optimal control.
Contrast with closed-loop methods (finding a policy).
Open-loop methods: indirect (derive optimality conditions) vs. direct (discretize then optimize).

58:32 Indirect Methods in Optimal Control

Define and determine necessary optimality conditions for optimal control problems.
Solve these conditions to filter candidates for optimal solutions.
These conditions are differential equations for infinite-dimensional problems.

1:00:06 Infinite-Dimensional Optimization

Optimal control problems are infinite-dimensional because the control 'u(t)' is a function (signal) over time.
Objective is to optimize a full signal u(t) over an interval [t0, tf].
Indirect methods are also known as 'optimize then discretize'.

1:08:23 Introduction to Calculus of Variations

Generalizes calculus to find extrema of functionals (functions of functions).
Goal: derive necessary optimality conditions for infinite-dimensional problems.
Mapping finite-dimensional intuition to infinite-dimensional calculus of variations.

1:11:54 Functionals, Linearity, and Norms

Functional J(x): assigns a real number to each function x in a class.
Example: J(x) = integral(g(x(t), x'(t), t) dt).
Norms define 'closeness' between functions, analogous to distance between vectors.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Stanford Online.