CSci574 / AI 520 Machine Learning: Classroom Session 09/15/2026
Watch on YouTube →
Overview
Derek Harter reviews Assignment 2 expectations, then introduces linear regression using a 47-house dataset to explain fitted lines, model parameters, vectorized NumPy predictions, and cost functions. He connects slope-intercept notation to parameter vectors and dummy features, compares absolute and squared errors, and prepares students to implement linear regression and gradient descent in Assignment 3.
Key takeaways
- For the 47-house example, a degree-1 linear regression uses square footage as its feature and produces one fitted price prediction for each house.
- A feature matrix needs a column of ones to represent the intercept as a dot product; with n features, the parameter vector contains n coefficients plus the bias.
- Summing signed prediction errors can be misleading because overpredictions and underpredictions cancel; MAE or MSE measures error magnitude instead.
- In the housing example, MAE is approximately $52,000 and RMSE approximately $64,000, illustrating that different regression metrics summarize errors differently.
- Adding the feature most correlated with Rain Tomorrow reportedly increased Assignment 2 classifier accuracy from roughly 85 to 87 percent to about 95 to 96 percent.
- Assignment 3 asks students to implement linear regression and gradient descent, making a correctly configured Python environment and passing pytest tests practical prerequisites.
Chapters
- Derek Harter asks students to make separate Git commits at minimum for each assignment task that implements a required function.
- He recommends making more, smaller commits and writing commit messages with a clear title and description.
- Assignment 3 is available, and the next two assignments focus on linear regression.
- Assignment 2 notebooks include data exploration with pandas methods such as `info` and `describe`, as well as visualizations and explanatory exercises.
- For the evaporation example, students used temperature as the single feature and plotted the fitted line from its slope and intercept.
- A straight regression line can be drawn from its predictions at two selected x-values, such as the minimum and maximum temperatures.
- Harter advises against installing packages or changing Python paths inside notebooks; students should configure their Python environment and Jupyter kernel correctly.
- The course dev containers configure Python packages under `/opt` and set `PYTHONPATH` to the source root so imports and tests work as intended.
- Students can run tests from the command line with Python's `-m pytest` option and should read the supplied tests rather than modify them.
- An Assignment 2 imputation test expected pandas Series output, including zero missing-value sums for the relevant filled features.
- Harter notes that adding the feature most correlated with the Rain Tomorrow target could improve the classifier.
- The baseline accuracy was approximately 85 to 87 percent; including that feature commonly raised results to about 95 to 96 percent.
- He recommends checking the dataset dictionary to identify what the pressure-related feature measures.
- Multiclass classification predicts one of three or more categories, as in MNIST digit recognition across labels 0 through 9.
- Some scikit-learn estimators handle multiclass tasks through one-versus-rest or one-versus-one classifiers behind the scenes.
- Multilabel classification returns multiple labels, while multi-output classification combines multiple outputs when at least one output has multiple classes.
- The example uses 47 house sales to predict price from house size in square feet.
- The plotted data show a positive association: larger houses generally have higher sale prices.
- Prices span roughly $175,000 to $700,000, making a linear model a reasonable first approximation despite possible nonlinearity.
- Two points uniquely determine a line, with slope equal to the change in price divided by the change in square footage.
- The slope describes the predicted price change for each additional square foot; the intercept is the predicted price at zero square feet.
- Using two selected sales gives an example slope of roughly 0.13 thousand dollars per square foot and an intercept around 107.67 thousand dollars.
- A line can be plotted by evaluating its equation at two x-values, including zero to show the intercept.
- With three or more data points, the points may not lie on one line, so a fitting rule is needed to define the best line.
- Lines through selected pairs can have very different slopes, including a negative slope that poorly reflects the overall house-size trend.
- The best-fitting line may pass through none of the observed points because it balances errors across the dataset.
- NumPy's `polyfit` with degree 1 computes a best-fit line and should match a linear regression implementation with compatible defaults.
- A degree-2 polynomial has three parameters, for the constant, x, and x-squared terms; higher degrees add further terms.
- For the 47-house example, the degree-1 fitted line has a lower intercept than the line through the initially selected two points.
- Harter notes that higher-degree polynomials will be useful for modeling nonlinear patterns in later coursework.
- Harter uses theta parameters to represent model coefficients: theta zero for the intercept and theta one for the feature coefficient.
- A degree-2 polynomial extends the parameter set to theta zero, theta one, and theta two; the constant term is also called a bias.
- A linear model can include multiple house features, such as square footage, bathrooms, room count, and age.
- With n features, the model has n feature coefficients plus an intercept parameter.
- Representing features and parameters as vectors allows predictions to be computed with a dot product.
- To include theta zero in that dot product, add a dummy feature x zero whose value is 1 for every example.
- Harter contrasts library defaults: scikit-learn generally includes an intercept, while statsmodels may require adding a constant feature explicitly.
- The resulting feature vector and parameter vector must have matching lengths.
- The house-size dataset becomes a 47-by-2 feature matrix after adding a column of ones for the intercept.
- The parameter vector contains the fitted intercept and slope, and matrix multiplication produces one prediction per house.
- Harter shows equivalent dot-product orientations, such as multiplying the feature matrix by theta or theta by the transposed matrix.
- NumPy returns a one-dimensional array of 47 predicted prices.
- Using the fitted theta values yields a predicted price for each of the 47 observed house sizes.
- Some predictions are close to actual sale prices, while others are substantially too high or too low.
- Real-world noise and unobserved factors affecting house prices mean a fitted model will rarely predict every training example perfectly.
- Prediction errors provide the basis for measuring how well a model fits the data.
- A cost function measures model quality by comparing predicted values with true labels across the dataset.
- Positive and negative errors can cancel when summed, so the near-zero total error for the fitted line does not mean its predictions are perfect.
- Mean absolute error uses the average magnitude of errors; the example's MAE is about $52,000.
- Mean squared error squares each prediction error before summing, and root mean squared error returns the result to the target's units; the example RMSE is about $64,000.
- Harter says regression work in the course will commonly use MSE or RMSE, and notes that Python-style sample indexing runs from 0 through m minus 1.
Transcript excerpt
0:27 Okie do. Uh few people here. Let's go and get started. Uh good afternoon. Um so, [clears throat] uh I did return back the assignment two evaluations. Um although I'm still working with a few people on some things. Uh but um yeah, I was going to review that real quickly. I won't spend too much time on that but let let me know you know as usual if you have questions on these um I didn't post a solution but I'll go through solution here um and um um I can as usual post those if everybody needs to see in more detail uh particular thing so I might do that like on the next assignment there might be some stuff um uh but but yeah once we go over the assignment two I think I'm going to jump right to kind of talking about our content this week uh
1:14 so our our next two units It's we're kind of focused going to be looking at linear regression, gradient descent. So I I this is kind of the heart of the course coming up. So if there's one thing in particular I hope you guys pay attention to is this. So we'll be getting into kind of the details of how one of the most basic machine learning algorithms work. So linear and logistic regression. We'll look at uh how uh what it means to like fit a model to a set of data and um a little bit of the mathematics behind it and that kind of stuff. So uh so I yeah I do like this uh and the next two assignments kind of are about linear regression. So we'll actually be implementing some of the stuff from scratch on the assignment three which should be visible now. Uh
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Derek Harter.