CPBS 7620 - Lecture 7 - Linear Functions

Overview

Nicholas Dwork develops the linear algebra behind linear functions, connecting vector spans, bases, matrix images, rank, and nullity. He then shows how orthogonalization supports projection onto a span, why linear functions are convex, and how the transpose identity implies that the null space of Aᵀ is perpendicular to the column space of A.

Key takeaways

For F(X) = AX, the image of the function is exactly the span of A's columns, making the matrix's column space a direct description of all attainable outputs.
The rank of A counts the dimensions in its image, while its nullity counts the dimensions in its null space.
A basis combines two requirements, linear independence and spanning; every basis of a given vector space has the same number of vectors.
Gram-Schmidt orthogonalization constructs perpendicular directions by subtracting prior projections, allowing a vector's projection onto a span to be computed as a sum of projections onto orthogonal directions.
Linearity implies convexity because F((1 − γ)X + γY) = (1 − γ)F(X) + γF(Y) for γ between 0 and 1.
Every vector in null(Aᵀ) is perpendicular to every column of A, so the null space of the transpose is the orthogonal complement of A's image.

Chapters

0:00 Vector Spans, Matrix Images, and Dimensions

Vector addition and scalar multiplication operate componentwise on vectors in Rⁿ.
The span of vectors is the set of their linear combinations; in R³, two linearly independent vectors span a plane through the origin.
For F(x) = Ax with A an m-by-n matrix, the image of F is the span of A's columns, also called its column space.
Rank is the dimension of the image, nullity is the dimension of the null space, and a basis is a linearly independent spanning set whose size gives the space's dimension.

4:13 Orthogonal Projection, Linear Maps, and Convexity

Orthogonal vectors have pairwise dot products of zero; an orthonormal set also has unit-length vectors, and an orthogonal matrix has orthonormal columns.
For projection onto the span of X₁ and X₂, set U₁ = X₁ and U₂ = X₂ minus its projection onto U₁, then add the projections of V onto U₁ and U₂.
A linear function satisfies additivity, F(X + Y) = F(X) + F(Y), and homogeneity, F(kX) = kF(X); every linear map from Rⁿ to Rᵐ can be represented as F(X) = AX.
For a real-valued linear function, convexity's inequality holds with equality, so every linear function is convex.

6:32 The Matrix Transpose and Orthogonality of Null Space and Column Space

Transposing an m-by-n matrix A swaps its rows and columns, so each entry satisfies (Aᵀ)ᵢⱼ = Aⱼᵢ.
The transpose identity (AX)·Y = X·(AᵀY) relates matrix multiplication to dot products.
If V lies in the null space of Aᵀ, then AᵀV = 0 means V has zero dot product with every column of A.
Consequently, null(Aᵀ) is perpendicular to the image of A, which is the column space of A.

Transcript excerpt

0:00 Hi, I'm Nicholas Twork, and today we're going to discuss linear functions. Here's how we're going to define vector addition and scalar multiplication. So, for adding two vectors A + B that equals the vector A1 A2 up to AN + B1 B2 up to BN, and that's going to equal a pointwise addition A1 + B1 A2 + B2 up to AN + BN. If we take a scalar and multiply the vector A, that's going to equal K * A1 A2 up to AN. And we're just going to multiply each component by the scalar, so it'll be KA1 KA2 up to KAN. Let A1 through AN be vectors of real numbers of size M. The span of A1 A2 through AN is the set of linear combinations of those vectors.

0:50 A vector space is a span of a set of vectors. The dimension of the vector space is the smallest number of vectors that generate the same span. The dimension of the span of A1 A2 through AN is N if and only if A1 A2 through AN are linearly independent. If they are linearly dependent, then the dimension will be less than N. Here's a geometrical understanding of what a span is. Here are two vectors in three dimensions. And if we take the span of these vectors, what we get is a plane through the origin. So, in 3D, two different vectors gives you a plane through the origin. In higher dimensions, a span gives you a

1:38 hyperplane through the origin. Let F be the function that maps RN to RM such that f(x) is equal to AX, where A is a matrix of real numbers of M rows and N columns. The image of F is the set AX where X is any real vector of size N.

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Nicholas Dwork.