27. Positive Definite Matrices and Minima

Overview

Positive definiteness of a real symmetric matrix means its quadratic form xᵀAx is positive for every nonzero vector, linking eigenvalues, leading principal determinants, pivots, and sums of squares. The lecture develops these tests from 2×2 examples to a 3×3 matrix, then connects positive definiteness to local minima, Hessians, and ellipsoid geometry through eigenvectors and eigenvalues.

Key takeaways

For a real symmetric matrix, positive definiteness is equivalent to all eigenvalues being positive and, by the leading-principal-minor test, all leading principal determinants being positive.
The matrix [[2,6],[6,18]] is positive semidefinite but not positive definite: its determinant is zero, its rank is one, and its eigenvalues are 0 and 20.
The quadratic form for [[2,6],[6,20]] is 2(x + 3y)² + 2y²; positive elimination pivots become the positive coefficients in this sum-of-squares representation.
At a stationary point of a multivariable function, a positive-definite Hessian guarantees positive curvature in every direction and identifies a strict local minimum.
For the 3×3 tridiagonal example, leading principal determinants 2, 3, and 4 yield positive pivots 2, 3/2, and 4/3, establishing positive definiteness without computing eigenvalues.
For a positive-definite symmetric matrix, eigenvectors determine ellipsoid principal directions, while eigenvalues determine curvature and semiaxis lengths proportional to their inverse square roots.

Chapters

0:00 Three Tests for Positive Definiteness in the 2×2 Case

For a real symmetric matrix, positive definiteness can be tested by checking that every eigenvalue is positive.
The determinant test checks positive leading principal minors, meaning the determinants of the upper-left 1×1, 2×2, and larger submatrices.
The pivot test requires positive elimination pivots; for a 2×2 matrix, the second pivot is the determinant divided by the first.

4:00 The Borderline Matrix with Eigenvalues 0 and 20

For the matrix [[2,6],[6,c]], positive definiteness requires c > 18 because its determinant is 2c − 36.
At c = 18, the determinant is zero, the matrix has rank one, and its eigenvalues are 0 and 20, making it positive semidefinite rather than positive definite.
The c = 18 matrix has only one pivot, illustrating why strict positive-pivot tests exclude singular positive-semidefinite matrices.

7:30 The Quadratic Form xᵀAx Defines Positive Definiteness

For A = [[2,6],[6,18]] and x = (x₁,x₂), multiplication gives xᵀAx = 2x₁² + 12x₁x₂ + 18x₂².
A symmetric matrix’s diagonal entries become pure-square coefficients, while each off-diagonal entry contributes twice to the mixed term.
Positive definiteness means xᵀAx > 0 for every nonzero x; the value is necessarily zero at x = 0.

12:00 A Negative Pivot Produces a Saddle, Not a Minimum

Changing the matrix to [[2,6],[6,7]] gives determinant −22 and pivots 2 and −11, so the matrix is indefinite.
Its quadratic form, 2x₁² + 12x₁x₂ + 7x₂², is negative at (1,−1), where it evaluates to −3.
The graph rises in some directions and falls in others, creating a saddle point at the origin rather than a bowl-shaped minimum.

19:00 Positive Eigenvalues Make the Origin a Bowl-Shaped Minimum

For A = [[2,6],[6,20]], the determinant is 4 and the trace is 22, so the eigenvalues must both be positive.
The quadratic function has zero first derivatives at the origin, while positive curvature in every direction makes the origin a strict minimum.
The comparison with a saddle emphasizes that vanishing first derivatives alone do not establish a minimum; the second-derivative behavior matters.

26:30 Completing the Square Reveals Positivity and Ellipse Level Sets

The form 2x² + 12xy + 20y² rewrites as 2(x + 3y)² + 2y², a sum of positive multiples of squares.
With the lower-right entry changed to 18, the final 2y² term disappears; with it changed to 7, the remainder becomes −11y², exposing indefiniteness.
Setting a positive-definite quadratic form equal to 1 gives an ellipse in two dimensions, while a saddle’s level curves are hyperbolas.

32:00 Gaussian Elimination Produces the Completing-the-Square Coefficients

Eliminating in [[2,6],[6,20]] uses multiplier 3 and produces pivots 2 and 2.
The multiplier 3 appears inside the square 2(x + 3y)², while the two pivots appear as the positive coefficients outside the squares.
For larger symmetric matrices, positive elimination pivots similarly produce a sum of squares with positive coefficients.

35:15 The Hessian Test Generalizes the Calculus Minimum Test

For a function of two variables, the Hessian is [[fxx,fxy],[fyx,fyy]], with fxy = fyx when the mixed partial derivatives agree.
A local minimum requires zero first derivatives and a positive-definite Hessian; in two dimensions this includes fxx > 0 and fxxfyy − fxy² > 0.
The matrix criterion extends the single-variable second-derivative test to functions of many variables.

38:00 A 3×3 Example Passes the Minor, Pivot, and Eigenvalue Tests

For the tridiagonal matrix [[2,−1,0],[−1,2,−1],[0,−1,2]], the leading principal determinants are 2, 3, and 4.
Its pivots are 2, 3/2, and 4/3, all positive, and its eigenvalues are 2 − √2, 2, and 2 + √2.
The associated form is 2x₁² + 2x₂² + 2x₃² − 2x₁x₂ − 2x₂x₃; positive pivots let it be represented as a positive sum of three squares.

45:00 Eigenvectors Set Ellipsoid Axes and Eigenvalues Set Their Lengths

A positive-definite quadratic form in three variables has ellipsoidal level sets, such as xᵀAx = 1, in place of the two-dimensional ellipse.
For a symmetric matrix, the principal-axis decomposition is A = QΛQᵀ, where Q contains orthonormal eigenvectors and Λ contains eigenvalues.
The eigenvectors give the ellipsoid’s principal directions, and its semiaxis lengths at level 1 are proportional to 1/√λ for the corresponding positive eigenvalues.

Transcript excerpt

0:05 OK, this is the lecture on positive definite matrices. I made a start on those briefly in a previous lecture. One point I wanted to make was the way that this topic brings the whole course together, pivots, determinants, eigenvalues, and something new- four plot instability and then something new in this expression, x transpose Ax, actually that's the guy to watch in this lecture. So, so the topic is positive definite matrix, and what's my goal?

0:53 First, first goal is, how can I tell if a matrix is positive definite? So I would like to have tests to see if you give me a, a five by five matrix, how do I tell if it's positive definite? More important is, what does it mean? Why are we so interested in this property of positive definiteness? And then, at the end comes some geometry. Ellipses are connected with positive definite things. Hyperbolas are not connected with positive definite things, so we- it's this, we, there's a geometry too, but mostly it's linear algebra and -- this application of how do you recognize 'em when you have

1:43 a minim is pretty neat. OK. I'm gonna begin with two by two. All matrices are symmetric, right? That's understood; the matrix is symmetric, now my question is, is it positive definite? Now, here are some -- each one of these is a complete test for positive definiteness. If I know the eigenvalues, my test is are they positive? Are they all positive? If I know these -- so, A is really -- I look at that number A, here, as the, as the one by one determinant, and here's the two by two determinant. So this is the determinant test.

2:35 This is the eigenvalue test, this is the determinant test. Are the determinants growing in s- of all, of all end, sort of,

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT OpenCourseWare.