CSci574 / AI 520 Machine Learning: Classroom Session 11/17/2026
Overview
Derek Harter introduces unsupervised learning and dimensionality reduction, explaining how high-dimensional, sparse data can make machine-learning models harder to generalize and how PCA projects data onto lower-dimensional axes that retain the most variance. He also reviews course project expectations, contrasts linear projection with manifold learning, and begins a scikit-learn PCA demonstration before technical difficulties interrupt the session.
Key takeaways
- PCA reduces dimensions by finding orthogonal axes ordered by variance, allowing a model to retain the directions with the greatest spread in the data.
- High-dimensional datasets can be sparse even when each feature is individually well-behaved, which can leave machine-learning algorithms without enough nearby examples to generalize.
- Adding training data can reduce high-dimensional sparsity, but projection or feature selection offers an alternative when acquiring more observations is not feasible.
- Feature-importance analysis and PCA are different reduction strategies: feature selection keeps chosen original variables, while PCA creates new coordinates from combinations of the original features.
- Manifold learning can represent curved lower-dimensional structures that a flat PCA projection may not capture, and scikit-learn provides implementations of both approaches.
- For the class project, readable notebooks, evidence of incremental progress, and documented environment setup help make analysis reproducible and understandable.
Chapters
- Derek Harter says Assignment 6 grading is delayed, Assignment 7 still needs to be posted, and its tentative deadline is the Sunday of Thanksgiving break.
- Test 2 is planned for the last week of classes, while the class project will be reviewed during finals week.
- Project repositories should show incremental work and use readable Jupyter notebooks for data exploration, cleaning, and modeling, with written explanations of decisions.
- Document reproducible environments with a requirements file and a README describing setup, notebook order, and any raw-to-clean data steps.
- Supervised learning trains a mapping from inputs to known labels and aims to generalize to unseen examples.
- Unsupervised learning works without supplied labels and can reveal structure in input data.
- K-means clustering can identify natural groupings in unlabeled data that may suggest classes for a later supervised-learning task.
- The course's upcoming unsupervised-learning topics are dimensionality reduction and clustering.
- A dataset's dimensionality corresponds to its number of input features; the Iris examples have four features, sometimes reduced to petal length and width.
- Each 28-by-28 MNIST image has 784 pixel features, while practical datasets can contain thousands or more features.
- High-dimensional spaces are difficult to visualize and behave less intuitively than two- or three-dimensional examples.
- Harter introduces dimensionality reduction as a way to simplify learning when many input dimensions make modeling difficult.
- In a high-dimensional unit hypercube, a random point is increasingly likely to lie near a boundary in at least one dimension.
- As feature count rises, sampled observations tend to be sparse and far apart, leaving many algorithms with too little local evidence to generalize well.
- More training data can fill in the feature space and improve regression approximations or classification decision boundaries.
- When collecting more data is impractical, reducing the number of dimensions can make the learning problem more manageable.
- Feature-importance analysis can identify a smaller subset, such as the top 10 to 20 percent of features, for comparison against models using all features.
- Removing irrelevant or weak features can sometimes maintain or improve model performance.
- Projection instead maps the original data into a new, lower-dimensional space.
- PCA is presented as a projection method that finds a lower-dimensional plane intended to lose as little useful information as possible.
- A standard projection such as PCA maps data onto a plane or hyperplane in a lower-dimensional space.
- Curved data structures may not be represented well by a flat plane, motivating manifold-learning methods.
- Manifold learning assumes the observations lie on a lower-dimensional structure embedded in a higher-dimensional space.
- Harter notes that scikit-learn includes PCA and manifold-learning techniques, though the course assignment focuses mainly on PCA.
- Harter begins a two-dimensional-to-one-dimensional example using a rotated set of points and a scikit-learn PCA fit.
- PCA selects the first principal axis in the direction of greatest variance, then selects subsequent axes orthogonal to earlier ones.
- The transformed coordinates express the data along these new axes, with the first component capturing the largest spread.
- The demonstration is introductory; Harter postpones the mathematical details of PCA to the next class.
- Around 37 minutes, Harter pauses the explanation and encounters missing figures and apparent notebook inconsistencies.
- He switches back to the textbook to troubleshoot but does not complete the PCA derivation or demonstration.
- The session ends with a short break while Harter checks the technical issue, consistent with the uploader's note that the session was cut short.
Transcript excerpt
0:15 All right. Um, [clears throat] so hello. Uh, welcome. I've got a few people here at the moment. I'm gonna go ahead and start. Uh, yeah. So, um I've gotten um a little bit behind. I don't I didn't return back the most recent assignment that everybody should have done. I hope everybody got that in. Uh if if not, I mean, you could maybe still do a little bit on it. Uh but uh I'll try and start working on that and get that back. Um looking forward, just remember uh I had I had a bad announcement up here too. I don't know if anybody saw that or not. So, uh we're we're kind of going to put up this unit on the chapter on dimensionality reduction sort of this week. Uh and then there's another one on uh principal uh
1:03 oh no on um like uh unsupervised learning is what it's called, but uh does a little bit with uh clusterings stuff like that. I'll post that one. that'll be kind of over uh like the half Thanksgiving week which is coming up next week and then the full week after that and then we're kind pretty much done with classes for the most part. So just keep that in mind. U I haven't I also haven't posted the assignment seven link yet. So I'll try and get that up as well. Um right now I've I've got a due date on that one for it would be the Sunday during the Thanksgiving break. I might give some more time on that because basically, you know, everybody should be aware, you know, we've got that assignment seven. There is going to be a test two which I'm probably planning on having open up during uh the
1:51 week of the last week of classes. Uh and
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Derek Harter.