28. Similar Matrices and Jordan Form
Watch on YouTube →
Overview
The lecture connects positive definite matrices to least squares, showing that AᵀA is positive definite exactly when a rectangular A has full column rank, and that positive definiteness is preserved by inversion and addition. It then defines similarity as B = M⁻¹AM, proves similar matrices share eigenvalues, and introduces Jordan form as the canonical structure that extends diagonalization to matrices with repeated eigenvalues and too few eigenvectors.
Key takeaways
- For any rectangular A, xᵀAᵀAx = ‖Ax‖², so AᵀA is positive semidefinite and becomes positive definite exactly when A has full column rank.
- If A is symmetric positive definite, then A⁻¹ is also positive definite because its eigenvalues are the positive reciprocals of A's eigenvalues.
- Similar matrices B = M⁻¹AM have identical eigenvalues, while each eigenvector x of A corresponds to the eigenvector M⁻¹x of B.
- Repeated eigenvalues do not guarantee diagonalizability: 4I and [[4, 1], [0, 4]] both have eigenvalues 4 and 4, but only the first has two independent eigenvectors.
- Jordan block sizes distinguish similarity classes even when eigenvalues and the number of independent eigenvectors match, as with a 3-plus-1 block structure versus two 2-by-2 blocks.
- Jordan form generalizes diagonalization to every square matrix, but it is numerically fragile because small changes in matrix entries can change eigenvalues and ranks.
Chapters
- A positive definite matrix is symmetric and has positive eigenvalues.
- The eigenvalues of A⁻¹ are reciprocals of A's eigenvalues, so a symmetric positive definite A has a symmetric positive definite inverse.
- Eigenvalues provide a direct way to verify positive definiteness when pivot behavior is not convenient.
- For positive definite A and B, xᵀ(A + B)x = xᵀAx + xᵀBx is positive for every nonzero x, so A + B is positive definite.
- In least squares, a rectangular m-by-n matrix A leads to the square, symmetric matrix AᵀA.
- The quadratic-form test is useful when the eigenvalues or pivots of a matrix combination are not known directly.
- The quadratic form xᵀAᵀAx equals (Ax)ᵀ(Ax), the squared length of Ax, so it can never be negative.
- AᵀA is positive definite precisely when Ax = 0 has only the zero solution, which is equivalent to A having full column rank.
- For an 11-by-5 A, independent columns give rank 5 and make AᵀA invertible, supporting the least-squares equations.
- Positive definite matrices are computationally favorable: elimination avoids row exchanges and zero pivots.
- Two n-by-n matrices A and B are similar when B = M⁻¹AM for some invertible M.
- If A has a full set of eigenvectors in S, choosing M = S produces the diagonal eigenvalue matrix Λ.
- For A = [[2, 1], [1, 2]], the eigenvalues are 3 and 1; other matrices formed by M⁻¹AM retain those eigenvalues.
- Similarity groups matrices into families, with a diagonal matrix representing the simplest member when diagonalization is possible.
- Starting from Ax = λx and substituting B = M⁻¹AM gives B(M⁻¹x) = λ(M⁻¹x).
- Every eigenvalue λ of A is therefore an eigenvalue of B, and the corresponding eigenvector transforms from x to M⁻¹x.
- Diagonalization changes the eigenvectors into coordinate vectors such as (1, 0) and (0, 1), while preserving the eigenvalues.
- Repeated eigenvalues can leave a matrix without enough independent eigenvectors to diagonalize.
- The matrix 4I is a singleton similarity family: M⁻¹(4I)M = 4I for every invertible M.
- The matrix [[4, 1], [0, 4]] also has eigenvalues 4 and 4, but only one independent eigenvector, so it is not similar to 4I.
- The closest canonical form for this non-diagonalizable case keeps a 1 above the diagonal, producing a Jordan block.
- A 2-by-2 matrix with trace 8 and determinant 16 has characteristic roots 4 and 4.
- Matrices such as [[5, 1], [-1, 3]] illustrate repeated eigenvalues without being the diagonal matrix 4I.
- Nondiagonalizable matrices with eigenvalues 4 and 4 can be similar to the Jordan block [[4, 1], [0, 4]], even when their entries differ.
- Jordan form serves as a canonical near-diagonal representative when a diagonal representative is impossible.
- A 4-by-4 matrix with zeros on the diagonal and two superdiagonal ones has eigenvalue 0 repeated four times, rank 2, and a two-dimensional null space.
- Changing a superdiagonal 1 to 7 preserves the rank and eigenvalue count, and does not change the Jordan structure.
- Moving the nonzero entries can create a 3-by-3 Jordan block plus a 1-by-1 block, rather than two 2-by-2 blocks.
- Both arrangements have two independent eigenvectors, but their different block sizes mean the matrices are not similar.
- Jordan’s theorem states that every square matrix is similar to a Jordan matrix made of Jordan blocks.
- Each block has one repeated eigenvalue on the diagonal, zeros below it, and ones immediately above it.
- The number of Jordan blocks equals the number of independent eigenvectors, while block sizes encode additional structure not captured by that count alone.
- When all n eigenvalues are distinct, the Jordan form is diagonal; computing general Jordan forms is numerically delicate because small changes can alter eigenvalues and ranks.
Transcript excerpt
0:08 Okay. This lecture is mostly about the idea of similar matrixes. I'm going to tell you what that word similar means and in what way two matrixes are called similar. But before I do that, I have a little more to say about positive definite matrixes. You can tell this is a subject I think is really important and I told you what positive definite meant -- it means that this -- this expression, this quadratic form, x transpose I x is always positive. But the direct way to test it was with eigenvalues or pivots or determinants. So I -- we know what it means, we know how to test it,
0:58 but I didn't really say where positive definite matrixes come from. And so one thing I want to say is that they come from least squares in -- and all sorts of physical problems start with a rectangular matrix -- well, you remember in least squares the crucial combination was A transpose A. So I want to show that that's a positive definite matrix. Can -- so I -- I'm going to speak a little more about positive definite matrixes, just recapping -- so let me ask a question. It may be on the homework. Suppose a matrix A is positive definite. I mean by that it's all --
1:48 I'm assuming it's symmetric. That's always built into the definition. So we have a symmetric positive definite matrix. What about its inverse? Is the inverse of a symmetric positive definite matrix also symmetric positive definite? So you quickly think, okay, what do I know about the pivots of the inverse matrix? Not much. What do I know about the eigenvalues of the inverse matrix? Everything, right? The eigenvalues of the inverse are one over the eigenvalues of the matrix.
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT OpenCourseWare.