33. Left and Right Inverses; Pseudoinverse
Watch on YouTube →
Overview
The lecture organizes inverses for an m-by-n matrix A by rank: a square full-rank matrix has a two-sided inverse, full column rank permits a left inverse, and full row rank permits a right inverse. For the general rank-r case, the pseudoinverse A+ reverses A between its row space and column space; the singular value decomposition constructs it by reciprocating nonzero singular values, with the resulting products acting as projections.
Key takeaways
- A two-sided inverse requires a square full-rank matrix: for an m-by-n matrix, rank must satisfy r=m=n.
- When A has full column rank, (A transpose A)^-1 A transpose is a left inverse, and A transpose A is the invertible coefficient matrix in the least-squares normal equations.
- When A has full row rank, A transpose (A A transpose)^-1 is a right inverse; Ax=b is solvable for every b, with n-m free variables.
- For any rank-r matrix, A restricted to the row space is a bijection onto the column space, and A+ reverses that restricted mapping.
- Using A=U Sigma V transpose, the pseudoinverse is V Sigma+ U transpose, where Sigma+ reciprocates nonzero singular values and leaves zero singular values at zero.
- A A+ and A+ A are projections onto the column space and row space, respectively, so the pseudoinverse removes components in the associated null spaces rather than restoring them.
Chapters
- The lecture reviews inverse types through the four fundamental subspaces: row space, null space, column space, and null space of A transpose.
- A two-sided inverse exists when rank r equals both the number of rows m and columns n, so A is square and full rank.
- The pseudoinverse is introduced as a generalization for rectangular or rank-deficient matrices, though it is identified as outside the course final exam scope.
- For full column rank, r=n and the null space of A contains only the zero vector because its n columns are independent.
- The system Ax=b has at most one solution: a consistent right-hand side has a unique solution, while an inconsistent one has none.
- A transpose A is an invertible n-by-n matrix, giving the left inverse (A transpose A)^-1 A transpose.
- Multiplying this left inverse by A yields the n-by-n identity, the same invertible matrix used in the normal equations for least squares.
- For full row rank, r=m and the rows are independent, so the null space of A transpose contains only zero.
- Every right-hand side b is attainable because elimination produces no zero row, but solutions are generally nonunique.
- The null space of A has dimension n-m, corresponding to n-m free variables and infinitely many solutions to Ax=b.
- A A transpose is invertible, and A transpose (A A transpose)^-1 is a right inverse whose product with A is the m-by-m identity.
- For full column rank, multiplying A by its left inverse gives A(A transpose A)^-1 A transpose, the projection onto the column space.
- For full row rank, multiplying the right inverse on the left of A gives A transpose (A A transpose)^-1 A, the projection onto the row space.
- These products are not identity matrices when A is rectangular; each acts as identity on its target subspace and removes components outside it.
- In the general case, rank r is smaller than both m and n, leaving nontrivial null spaces that prevent an ordinary inverse.
- A maps the r-dimensional row space one-to-one onto the r-dimensional column space, while annihilating components in the null space.
- The pseudoinverse A+ is defined as the inverse mapping from the column space back to the row space.
- If distinct vectors x and y lie in the row space and Ax=Ay, then x-y lies in both the null space and row space; their orthogonality forces x-y=0.
- Least-squares regression commonly relies on A transpose A being invertible, which requires full column rank.
- Repeated or dependent measurements can make the columns of A dependent and A transpose A singular.
- The pseudoinverse supplies a useful generalized solution when the full-column-rank assumption behind the usual least-squares formula fails.
- Write A as U Sigma V transpose, where U and V are orthogonal and Sigma contains r nonzero singular values among its zero entries.
- Construct Sigma+ by replacing each nonzero singular value sigma_i with 1/sigma_i and transposing the rectangular shape.
- The pseudoinverse is A+=V Sigma+ U transpose, since the orthogonal factors invert by transposition and Sigma+ handles the rank deficiency.
- Sigma Sigma+ is an m-by-m diagonal projection with r ones, onto the column-space coordinates; Sigma+ Sigma is an n-by-n projection onto the row-space coordinates.
- For A, the corresponding products A A+ and A+ A project onto the column space and row space, respectively, rather than producing full identity matrices.
- The SVD isolates the difficulty in Sigma, where zero singular values are left at zero and nonzero values are inverted; this gives a minimal natural inverse on the rank-r subspaces.
- The lecture closes by noting that the pseudoinverse appears in section 7.4 and is not expected on the final, while the rank and subspace reasoning is core course material.
Transcript excerpt
0:03 Yes, OK, four, three, two, one, OK, I see you guys are in a happy mood. I don't know if that means 18.06 is ending, or, the quiz was good. Uh, my birthday conference was going on at the time of the quiz, and in the conference, of course, everybody had to say nice things, but I was wondering, what would my 18.06 class be saying, because it was at the exactly the same time. But, what I know from the grades so far, they're basically close to, and maybe slightly above the grades that you got on quiz two. So, very satisfactory.
0:52 And, then we have a final exam coming up, and today's lecture, as I told you by email, will be a first step in the review, and then on Wednesday I'll do all I can in reviewing the whole course. So my topic today is -- actually, this is a lecture I have never given before in this way, and it will -- well, four subspaces, that's certainly fundamental, and you know that, so I want to speak about left-inverses and right-inverses and then something called pseudo-inverses.
1:39 And pseudo-inverses, let me say right away, that comes in near the end of chapter seven, and that would not be expected on the final. But you'll see that what I'm talking about is really the basic stuff that, for an m-by-n matrix of rank r, we're going back to the most fundamental picture in linear algebra. Nobody could forget that picture, right? When you're my age, even, you'll remember the row space, and the null space. Orthogonal complements over there, the column space and the null space of A transpose column, orthogonal complements over here. And I want to speak about inverses.
2:29 OK. And I want to identify the different possibilities.
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, MIT OpenCourseWare.