CSci574 / AI 520 Machine Learning: Classroom Session 09/08/2026
Overview
Derek Harter walks through the scikit-learn workflow for supervised learning and compares it with statsmodels, using linear regression for house prices and logistic regression for handwritten-digit classification. He explains feature and target data, model fitting and evaluation, train/test splits, regression metrics, confusion matrices, statsmodels intercept handling and statistical summaries, then reviews the weather-data tasks in Assignment 2.
Key takeaways
- In scikit-learn, supervised learning follows a reusable pattern: instantiate an estimator, fit it with X and y, then predict on data with the same feature columns.
- A train/test split helps assess generalization: the housing example reserves 30% of 5,000 samples for testing, and regression performance is compared using metrics such as RMSE.
- Logistic regression performs classification despite its name; the 8 by 8 digit example uses 64 pixel features and predicts one of 10 integer classes.
- Classification accuracy should be supplemented with a confusion matrix, because class-by-class errors are not visible from a single accuracy score.
- statsmodels requires an explicit constant column for an intercept, unlike scikit-learn's default behavior; with that adjustment, their linear-regression coefficients match.
- statsmodels can provide confidence intervals and class probabilities; for multiclass predictions, argmax converts the probability vector into the most likely label.
Chapters
- The class is working through Units 3 and 4, which cover supervised learning and the scikit-learn workflow.
- Assignment 2 applies both regression and classification, including linear regression and logistic regression.
- Regression predicts real-valued targets such as house prices; binary classification predicts discrete labels such as yes or no.
- After cleaning, input features are arranged in a two-dimensional matrix called X, with features as columns and samples as rows.
- The target vector y contains one value per sample, with real numbers for regression and integer category codes for classification.
- Multiclass targets are commonly encoded as integers such as 0 through 9 rather than one-hot vectors.
- Choose a scikit-learn estimator, create its model instance, set any desired hyperparameters, and call fit with features and labels.
- After fitting, use predict on new data with the same feature columns to evaluate generalization.
- Default hyperparameters provide a starting point, while estimator documentation explains available settings and methods.
- The housing example predicts individual house prices from features such as house age, room count, and average income.
- LinearRegression can be instantiated with default settings, then fitted using a pandas DataFrame or NumPy array.
- After fitting, coefficients and the intercept are available as model attributes; each feature has a corresponding coefficient.
- A fitted linear model has one coefficient per input feature plus an intercept term.
- Predictions can be compared with true house prices, but predictions on training samples alone do not establish generalization.
- A plot of predicted versus actual values shows deviations from perfect predictions as a cloud around the ideal straight line.
- A 70/30 split of 5,000 housing samples gives 3,500 training samples and 1,500 held-out test samples.
- Common splits include 70/30, 75/25, and 80/20; the best choice depends on dataset size and the need for both training data and reliable evaluation.
- Mean squared error and mean absolute error quantify regression errors; root mean squared error returns the result to the target's original units.
- Similar training and test errors suggest less overfitting, while a large test-error increase signals weaker generalization.
- Despite its name, logistic regression is a classification algorithm, not a method for predicting continuous values.
- The example uses 8 by 8 grayscale handwritten-digit images, flattened into 64 features, with labels from 0 through 9.
- The scikit-learn supervised-learning workflow remains the same, but the estimator must support classification.
- The LogisticRegression example changes the solver from the default LBFGS to Newton-CG, illustrating solver choice as a hyperparameter.
- Trying different solver and model settings can help identify configurations that perform better on a particular dataset.
- The multiclass model has coefficient sets and intercepts for each of the 10 digit classes, rather than one coefficient vector for all classes.
- Accuracy measures the fraction of correctly classified examples, making it suitable for classification but not naturally meaningful for regression.
- The digit classifier reaches 100% accuracy on training data and about 97% on held-out test data.
- A confusion matrix compares true labels with predicted labels; correct classifications lie on the diagonal, and a heat map makes errors easier to inspect.
- Accuracy alone can hide important classification weaknesses, so the confusion matrix provides class-by-class detail.
- statsmodels grew from statistical analysis and experimental-design use cases, often involving smaller samples than large-scale machine-learning datasets.
- Its summaries make statistical quantities such as confidence intervals readily available.
- The notebook imports statsmodels under the conventional alias SM and notes that it may need to be installed separately from NumPy and Matplotlib.
- A line with one feature requires a slope and an intercept; the intercept is an additional fitted parameter.
- scikit-learn includes the intercept by default, while statsmodels requires adding a constant column with add_constant.
- With the constant included, statsmodels OLS and scikit-learn LinearRegression produce matching coefficients for the housing example.
- statsmodels uses an OLS model object and fits it separately; its API also commonly places the target before the feature matrix.
- The fitted statsmodels result exposes parameters and an R-squared value, a commonly reported measure of regression fit.
- A higher R-squared generally indicates a stronger fit, but it should not be treated as a complete measure of model quality.
- The statsmodels summary reports confidence intervals, including a 95% interval for an estimated feature coefficient.
- For multiclass classification, statsmodels provides MNLogit, its multinomial logistic-regression model.
- The example may fail to converge on the small digit dataset, although fitted parameters and predictions can still be available.
- Unlike scikit-learn's class-label predictions, statsmodels can return a probability for each class; argmax selects the most probable class.
- A sample whose largest predicted probability corresponds to digit 5 is classified as a 5.
- Assignment 2 uses weather data for exploration, correlation analysis, and regression with both scikit-learn and statsmodels.
- The notebook includes required coding tasks and tests for the two libraries' linear-regression implementations.
- The assignment is due Sunday; Derek Harter advises students to accept it, begin the notebook work, and contact him if they encounter problems.
- The next class is planned to focus on classification materials.
Transcript excerpt
0:04 Okay. Uh hello. Uh is uh can you guys hear me okay online here? >> Yes, it is. >> Yes, professor. >> Great. Thank you. Perfect. Uh yeah, we have a lot of people online here. I'm going to do the usual. Um um but uh I'm still setting up here. Just give me one more moment. Almost ready. [clears throat] Okay. Uh so good afternoon. I'll go
1:00 ahead and start. Um might be a little short today. I've got something kind of happening here uh after uh but um so uh reminders as usual at the start here. So we're kind of on the uh unit three four. So kind of the second assignment. Um so you know I I don't know if I've mentioned this before. I kind of group these in sort of groups of two usually with one assignment. So um so so basically our unit three and four was sort of finishing up some introduction but we're really getting into the basics of sort of classification regression and uh well and and using scikitlearn was sort of the the big picture sort of thing that we looked at uh last time. So um so yeah, don't forget about the
1:48 assignment two. So my plan for today is um um we'll talk about assignment two. I think I'm going to though jump right to the uh kind of talking about this notebook where I talk have the uh some information about using the stats model as well as using scikitlearn for both regression and classification. So, so we haven't we haven't formally talked about I guess we did a little bit. We we we we talked about sort of the two main uh things you might be doing uh in machine learning which is you know
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Derek Harter.