Lecture 1: Introduction

Overview

Burton Ma introduces optimization as applied linear algebra and calculus for computing and machine learning, explaining how the course’s tests, resources, MATLAB examples, and historical foundations fit together. He traces techniques from Archimedes and Newton through least squares, Lagrange multipliers, gradient descent, and linear programming, then demonstrates a grid-search approximation to Fermat’s minimum-distance problem using MATLAB vectorization and contour plots.

Key takeaways

The course tests four topics-based assessments worth 16% each, allows two to be dropped with their weight transferred to the cumulative exam, and uses short written answers rather than multiple choice.
Burton Ma’s lectures and slides are the primary test reference; Dr. Randy Ellis’s notes and videos are supplementary resources, and MATLAB is used for examples rather than assessed programming.
Optimization links classical mathematics to machine learning: least squares, Lagrange multipliers, and gradient descent are examples of methods developed for problems that remain widely useful.
A brute-force solution to Fermat’s triangle problem approximates the minimizing point by evaluating the sum of distances from many grid candidates to the three vertices.
MATLAB’s `meshgrid`, reshaping, and element-wise array operators let the distance search run in vectorized form instead of explicit nested loops.
AI coding assistance can produce unusable code: MATLAB Copilot recognized Ma’s intended calculation but generated a line that did not run, making verification essential.

Chapters

0:00 Course Staff, Office Hours, and Learning Resources

Burton Ma introduces TAs Jack and Mac, who have prior experience with the course; Mac is away this term but reachable by email and Teams.
Office hours are held in Goodwin Hall on the seventh floor, beginning the following week; students can email Ma with course number 6371 to arrange another time.
There is no required textbook: Dr. Randy Ellis’s PDF notes and selected videos accompany Ma’s detailed lecture slides, which provide the material used for tests.

5:30 Four Tests, Dropped Scores, and the Cumulative Exam

The course has four tests worth 16% each; students may drop two, with the corresponding weight shifted to the exam, and there are no makeup tests.
Tests are scheduled for Thursdays at 10:30, use short written answers rather than multiple choice, and are not cumulative.
The final exam is cumulative because it can replace test marks; exceptional extended absences are handled individually, while missed exams follow university deferral policy.

10:22 MATLAB, Calculus Preparation, and Practice Homework

MATLAB is the course’s implementation tool for linear algebra and calculus examples, not a test subject; students will not be asked to write MATLAB answers on tests.
The course relies on differential calculus, especially the chain rule and Taylor series approximations; Ma says he will explain concepts students have not encountered or forgotten.
Ungraded homework includes solutions and is intended for practice; Ma recommends attempting problems before checking answers because the material is relevant to tests.

12:19 Optimization’s Role in Machine Learning and AI

The course title reflects a rebranding effort at the School of Computing to strengthen its AI and machine-learning curriculum; the underlying courses were not originally framed as AI courses.
Optimization is finding the best member of a set under specified conditions, such as maximizing a course grade while minimizing effort.
Applied linear algebra, calculus, and numerical algorithms prepare students to follow the mathematics in relatively recent machine-learning research papers.
The course includes techniques directly used in modern machine learning as well as historical methods such as support vector machines, which remained heavily researched into the 2000s and 2010s.

16:08 Ancient Geometry, Newton’s Fluid Problem, and the Brachistochrone

Archimedes studied the point on a parabolic arc farthest from a chord and derived the parabolic segment’s area as four-thirds the area of a related triangle.
Hero of Alexandria’s reflection problem minimizes the path from two points to a line; the solution has equal incoming and outgoing angles, as with a reflected ray of light.
Newton developed calculus methods while investigating fluid resistance on a sphere and a same-radius cylinder, a problem he connected to shipbuilding.
Johann Bernoulli’s brachistochrone asks which curve lets a ball travel between two points in the shortest time; the answer is a curved path rather than a straight line.

23:11 Least Squares, Lagrange Multipliers, and Optimization Algorithms

Gauss’s least-squares principle fits a line by minimizing the sum of squared vertical residuals, a method widely used across science, mathematics, and economics.
Joseph-Louis Lagrange developed Lagrange multipliers while studying the Moon’s libration; Ma says the course will examine the method in detail.
Augustin-Louis Cauchy described an iterative numerical approach now known as gradient descent, which can help solve nonlinear systems of equations.
John von Neumann proved the minimax theorem for finite two-player zero-sum games, while George Dantzig developed linear programming and the simplex method for constrained optimization.

31:18 Fermat’s Minimum-Sum-of-Distances Problem

Fermat’s problem asks for a point whose total distance to the three vertices of a non-collinear triangle is as small as possible.
Evangelista Torricelli produced a geometric solution in the 1640s; Ma introduces a brute-force numerical alternative that samples candidate points across the triangle.
The algorithm computes each candidate’s distances to vertices A, B, and C, then keeps the point with the smallest total.
Ma establishes course notation: points are represented by vectors, vectors are column vectors, and double vertical bars indicate distance.

36:11 Building a MATLAB Grid for the Triangle Search

Ma stores the triangle’s three two-dimensional vertices in a MATLAB matrix and uses plotting commands to draw the triangle.
Rather than constructing candidate points with nested loops, he uses `linspace` to create coordinate vectors and `meshgrid` to form a two-dimensional grid.
MATLAB is optimized for matrix and vector operations, so vectorized calculations can avoid the overhead of interpreted loops.
Blank areas in Ma’s slides are intentional spaces for live demonstrations and student note-taking.

42:11 Copilot, Reshaping, and Vectorized Distance Calculations

MATLAB Copilot inferred Ma’s intended distance calculation but generated code that did not run, so he warns students to verify AI-generated code rather than trust it blindly.
Ma reshapes triangle coordinates into a three-dimensional array so each grid point can be compared with all three vertices in a single vectorized operation.
Element-wise operators such as `.^` square corresponding array entries; combining squared x- and y-differences and taking square roots produces distances.
The vectorized computation performs the work of nested loops over grid coordinates and triangle vertices without writing those loops explicitly.

47:27 Summing Distances and Visualizing the Approximate Minimum

MATLAB’s `sum` operation along the third dimension adds each grid point’s distances to all three triangle vertices.
Using 101 coordinate samples between 0 and 5 and 101 between 0 and 4 gives a denser search grid than the earlier small demonstration.
The resulting contour plot shows curves of constant total distance and indicates where the minimum lies; Ma leaves the interpretation for the next class.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Burton Ma.