Linear Algebra - Lecture 19 - Fall, 2026
Overview
Lecture 19 develops the geometric foundations of Gram-Schmidt: Cauchy-Schwarz defines the angle between nonzero vectors, while orthogonal complements and projection decompose vectors into parallel and perpendicular parts. The Gram-Schmidt algorithm converts any basis into an orthogonal basis and then an orthonormal basis; for the basis (2,1), (1,2) in R², it produces (2,1) and (-3/5, 6/5), then normalizes them to (2/√5, 1/√5) and (-1/√5, 2/√5).
Key takeaways
- For nonzero vectors V and W, the inner product determines their angle through cos θ=⟨V,W⟩/(||V|| ||W||), with Cauchy-Schwarz guaranteeing a valid cosine value.
- The projection of V onto a nonzero direction W is (⟨V,W⟩/||W||²)W, and subtracting it leaves a vector orthogonal to W.
- Gram-Schmidt constructs each new orthogonal vector by subtracting its projections onto every previously constructed vector.
- Normalizing each nonzero vector in an orthogonal basis preserves orthogonality and yields an orthonormal basis.
- For the R² basis (2,1), (1,2), Gram-Schmidt yields the orthonormal basis (2/√5,1/√5), (-1/√5,2/√5).
- Coordinates in an orthonormal basis preserve the familiar dot-product calculation: the inner product of coordinate vectors x and y is Σxᵢyᵢ.
Chapters
- The Cauchy-Schwarz inequality bounds |⟨V,W⟩| by ||V|| ||W||, so the normalized inner product lies between -1 and 1.
- For nonzero vectors, define cos θ = ⟨V,W⟩/(||V|| ||W||), with θ in [0, π].
- In R², taking W along the x-axis recovers the familiar cosine formula x/√(x²+y²).
- For a subspace W, W⊥ consists of vectors whose inner product with every vector in W is zero.
- In R³, if W is the XY-plane, testing (a,b,c) against every (x,y,0) forces a=b=0, so W⊥ is the Z-axis.
- W⊥ is a subspace: bilinearity shows sums and scalar multiples of vectors orthogonal to W remain orthogonal to W.
- For nonzero W, the component of V parallel to W is V₁ = (⟨V,W⟩/||W||²)W.
- The perpendicular remainder is V₂ = V − V₁; expanding ⟨V₂,W⟩ gives zero.
- The formula also uses the unit vector u=W/||W||: the projection is ⟨V,u⟩u.
- Suppose V has two decompositions into a vector parallel to W and a vector orthogonal to W.
- The difference between the two parallel components is also orthogonal to W, yet is a scalar multiple cW.
- Since ⟨cW,W⟩=c||W||² and W is nonzero, c=0, proving both components are unique.
- An orthogonal set has pairwise perpendicular vectors; an orthonormal set also requires every vector to have length 1.
- For an orthonormal basis e₁,…,eₙ, write V=Σxᵢeᵢ and W=Σyⱼeⱼ.
- Because ⟨eᵢ,eⱼ⟩ is the Kronecker delta, ⟨V,W⟩ reduces to Σxᵢyᵢ, the standard coordinate dot product.
- Starting from a basis w₁,…,wₙ, retain the same spans of the first k vectors while constructing orthogonal vectors v₁,…,vₖ.
- At step k+1, subtract each projection onto the previously constructed vectors: vₖ₊₁=wₖ₊₁−Σᵢ₌₁ᵏ(⟨wₖ₊₁,vᵢ⟩/||vᵢ||²)vᵢ.
- Taking the inner product with any vⱼ makes all terms vanish except the jth projection, which cancels ⟨wₖ₊₁,vⱼ⟩.
- The resulting vector is orthogonal to v₁,…,vₖ, and the first k+1 vectors retain the same span as the original basis vectors.
- Once Gram-Schmidt produces an orthogonal basis v₁,…,vₙ, set uᵢ=vᵢ/||vᵢ||.
- Each uᵢ has length 1, and scaling preserves the zero inner products between distinct vectors.
- The normalized vectors therefore form an orthonormal basis for the same space.
- The original vectors have inner product 2·1+1·2=4, so they are not orthogonal.
- With w₁=(2,1), the second orthogonal vector is v₂=(1,2)−(4/5)(2,1)=(-3/5,6/5).
- The inner product of (2,1) and (-3/5,6/5) is -6/5+6/5=0.
- Normalizing (2,1) gives u₁=(2/√5,1/√5); normalizing (-3/5,6/5) gives u₂=(-1/√5,2/√5).
- These two unit vectors are perpendicular and form an orthonormal basis of R².
- The closing guidance emphasizes practicing the projection calculations and Gram-Schmidt homework, alongside understanding the proof.
Transcript excerpt
0:02 This is linear algebra. Lecture 19 and we're going to talk about the Gram Schmidt ortho normalization algorithm. So let's recall that that V be an n-dimensional
0:54 inner product space and the inner product has many beautiful properties. One is that it satisfies what is called the Cauchy-Schwarz inequality which says that for any two vectors V and W the inner product of V and W in absolute value is at most the length of V times the length of W. Another way of saying that is that the absolute value of V and W divided by the length of V times the length of W is at most one and of course greater than or equal to zero since these are non-negative.
1:42 And if you take away the absolute value signs this says that the inner product of V and W is over the length of V times the length of W. >> is between -1 and +1 because if something has absolute value of most one it's a it's a number between -1 and +1. And we might note that another function that you know well that has these two satisfies these inequalities is that for any angle theta the cosine of theta is also between -1 and one.
2:30 That you might recall, I hope you do that this is +1, this is -1 0 pi over 2 and pi this is the graph of the cosine of theta and in fact we can use the inner product to define the angle between two vectors V and W. So if this is V and this is W
3:17 not just in the plane but in some n-dimensional space we want to be able to define an angle between them and the definition of course once you know the cosine
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Nathanson math lectures.