CPBS 7620 - Lecture 10 - Pseudoinverse

Overview

Nicholas Dwork derives the Moore-Penrose pseudoinverse as a way to solve least-squares systems when A is not invertible: project B onto the image of A, then choose the smallest-norm solution that maps to that projection. He gives formulas for full-rank tall and short matrices, recommends numerically stable software routines over direct formulas, and demonstrates least-squares regression for predicting warfarin doses.

Key takeaways

For a noninvertible A, the least-squares output is the projection of B onto the image of A, because no vector AX can produce a point outside that image.
A nontrivial null space makes least-squares solutions nonunique: if X is a minimizer and AV = 0, then X + V gives the same output.
The pseudoinverse resolves that nonuniqueness by choosing the minimum-L2-norm solution, a useful criterion when large inputs carry a cost such as spacecraft thruster fuel.
For full-column-rank tall matrices, A† = (AᵀA)⁻¹Aᵀ; for full-row-rank short matrices, A† = Aᵀ(AAᵀ)⁻¹.
Although these formulas explain the pseudoinverse analytically, direct use can be numerically unstable; MATLAB backslash, numpy.linalg.lstsq, and LAPACK offer stable alternatives.
In the warfarin regression example, changing the target from dose D to its square root improved prediction, after which a new patient's estimate could be computed from a feature-vector dot product.

Chapters

0:00 Least Squares, Projections, and the Minimum-Norm Pseudoinverse

When A is not invertible, solve for X by minimizing the L2 norm of AX minus B.
Because every AX lies in the image of A, the closest attainable output to B is its projection onto that image.
If A has a nontrivial null space, adding any null-space vector to a minimizer leaves AX unchanged, so infinitely many least-squares solutions may exist.
The Moore-Penrose pseudoinverse selects the minimum-norm X that maps to the projection of B onto the image of A.

4:01 Full-Rank Pseudoinverse Formulas for Tall and Short Matrices

For a tall, full-column-rank matrix, the normal equations are AᵀAX = AᵀB, giving A† = (AᵀA)⁻¹Aᵀ.
For a short, full-row-rank matrix, every B is attainable; the minimum-norm solution lies in the image of Aᵀ.
Orthogonality between the image of Aᵀ and the null space of A shows that adding a null-space vector cannot reduce the solution norm.
For a short, full-row-rank matrix, A† = Aᵀ(AAᵀ)⁻¹, so X* = A†B.

11:10 Stable Least-Squares Tools and Warfarin Dose Regression

Direct full-rank pseudoinverse formulas are numerically unstable; Dwork recommends MATLAB backslash, Python numpy.linalg.lstsq, or LAPACK routines.
Least squares supports applications including linear regression, system identification, experiment planning, and data fusion.
A warfarin study encoded patient features including age, height, weight, gene variants, race, and medication use, then fit coefficients to predict stable dose.
The linear model predicted dose poorly, but predicting the square root of dose worked well; new-patient predictions use the fitted coefficient and feature dot product.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Nicholas Dwork.