Save this video free

Lecture 8b

Prof. Schniter · 47:02 · Watch on YouTube

Part of The Ohio State University ECE 5307. See all 30 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

The lecture explains stacking as a two-stage ensemble method: cross-validated predictions from diverse base estimators become meta-features for a trained combiner, often ridge regression or classification in scikit-learn. It then develops decision trees, covering recursive feature-threshold splits, regression and classification impurity measures, overfitting controls, and random forests, which reduce tree variance by training trees on randomized data and feature subsets.

Key takeaways

Chapters

0:00 Stacking Trains a Combiner on Base-Estimator Predictions
2:17 K-Fold Cross-Validation Creates Stacking Meta-Features
5:17 Decision Trees Partition Features into Prediction Regions
10:00 Recursive Splits Search Features and Thresholds
20:00 Car-Risk Example Builds a Three-Region Classification Tree
25:48 Regression Trees Select Splits by Weighted Error
29:26 Classification Trees Measure Impurity with Gini
35:48 Generic Losses and Decision-Tree Tradeoffs
44:28 Random Forests Reduce Decision-Tree Variance

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:01 Okay, next we'll talk about stacking. Um so in stacking uh what you do is you take different base estimators. So here we're getting diversity using different estimators. Um and we basically give them the same features and the same data for training. So no diversity from those perspectives. Um >> [clears throat] >> what we do is we we train these different base estimators to get these predictions. Uh we'll call them Y hat one through L. And then we um we have a separate combiner function here that is also something we train. So that's a little bit different than voting or averaging

0:50 where this is the combiner would just be a fixed deterministic procedure, but here we're actually going to going to train this. >> [clears throat] >> And the combiner um takes those L different predictions, one for each base estimator, and from them uh learns to predict the the final target. >> [clears throat] >> Now you could also sort of as an option you can also give the combiner uh the features as well. Uh but here uh in this picture we don't do that. Oh [clears throat] actually here yeah, so here's the optional. Um Now of course um because these predictors are using the same data and the same features, uh these

1:36 um predictions can be very correlated. And so a popular choice for the combiner is to use um ridge regularization. So you know, ridge if you you know, you can do this with classification or regression. And um in fact there's some uh nice um scikit-learn uh that include cross validation and ridge. So, you have ridge ridge CV for regression and ridge classifier CV for classification. Okay, so let's get into a little bit more detail about how we do this because

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Prof. Schniter.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.