10/26/26: Exam-02 Review
Watch on YouTube →
Overview
Joseph Zambreno reviews the material for Exam 02, which covers classification, data-centric machine learning, MLPs, ConvNets, and computer vision, with a 12-question, 80-minute format and one double-sided handwritten notes sheet. The review explains key concepts through practice questions, including PyTorch cross-entropy and label encodings, LeNet design choices, data leakage, convolutional edge detection, and why CNNs and pretrained models depend on meaningful spatial structure and fixed input dimensions.
Key takeaways
- PyTorch's built-in cross-entropy accepts integer class targets and logits because it handles the class selection and numerically stable log-sum-exp calculation internally; manual cross-entropy commonly uses one-hot targets.
- LeNet's ReLU and max-pooling choices address vanishing gradients and preserve strong local activations, while dropout primarily regularizes against overfitting rather than directly improving fit to the training task.
- Composing affine layers without an intervening nonlinearity still yields one affine transformation, so activations such as ReLU are what let deeper networks represent functions a single linear layer cannot.
- A CNN applied to an arbitrary 9-by-9 reshaping of 81 tabular features may learn artifacts of feature ordering, not meaningful spatial relationships.
- The apparent lower pneumonia risk for asthma patients arose from treatment patterns in the data, illustrating data leakage that can make a pre-treatment prediction model misleading.
- Image resolution affects the feature dimensions expected by classifier layers; pretrained AlexNet or ResNet-50 weights depend on maintaining compatible model inputs.
Chapters
- Exam 02 covers material completed since Exam 01: data-centric machine learning, MLPs, ConvNets, and computer vision applications.
- The associated labs are Labs 3 and 4; sequence-model labs are underway, but that material is not part of this exam review.
- Students are urged to reserve testing-center slots early because availability may be tighter than for the first exam.
- Exam 02 uses the Exam 01 format: 12 questions in 80 minutes, with conceptual explanations, code interpretation, and analysis of result graphs.
- Students may bring one double-sided, handwritten 8.5-by-11-inch notes sheet; pacing matters because some students ran out of time on Exam 01.
- Textbooks support review, but their end-of-chapter questions are less aligned with the exam than the course's sample questions and lecture guidance.
- A class survey ranks 12 practice questions by perceived challenge and helps Joseph Zambreno choose which to discuss.
- The informal results suggest that lab-derived code and result interpretation feel harder than conceptual questions.
- Zambreno notes that lab-specific memorization is not expected, and some questions can be reasoned through without having completed the exact lab.
- One-hot encoding represents a class among n possibilities with an n-element vector containing one 1 and otherwise 0s.
- Manual cross-entropy uses the one-hot target to select the true class's predicted probability; PyTorch's built-in cross-entropy accepts integer class labels directly.
- The built-in loss works from logits using numerically stable log-sum-exp calculations, so it does not require explicitly computing softmax or converting labels to one-hot form.
- For accuracy, argmax selects the index of the largest predicted score, which can be compared with an integer target such as y.long().
- The LeNet remix differs from the classical version in three ways: it adds dropout, replaces sigmoid with ReLU, and replaces average pooling with max pooling.
- Sigmoid gradients can become small and compound across layers, creating vanishing-gradient problems; ReLU helps preserve useful gradients for positive activations.
- Max pooling retains the strongest activation in a window, while average pooling can smooth away edge-like or otherwise prominent features.
- The question asks which choices help training; identifying dropout as a difference is not by itself an explanation of improved classification training.
- A dropout rate of 0.5 randomly zeros about half the relevant activations during training; at inference, dropout is disabled and the layer uses a scaled pass-through.
- Dropout reduces overfitting by preventing a model with many weights, especially in fully connected layers, from relying too heavily on particular units.
- LazyLinear infers its input dimension when it receives a batch, hiding the manual calculation of flattened feature sizes in models such as LeNet and AlexNet.
- Changing image resolution can change the flattened feature count and make a conventional nn.Linear layer's expected input dimensions incompatible.
- The reviewed graph plots binary cross-entropy loss against a logit for the two possible true labels.
- A confidently wrong prediction drives cross-entropy toward infinity because the calculation involves the logarithm of a probability approaching zero.
- The plotted range of 0.01 to 0.99 avoids evaluating log(0); practical implementations instead use numerical-stability techniques, including operations directly on logits.
- Composing multiple linear or affine transformations without an activation function is equivalent to one affine transformation.
- Multiplying weight matrices produces another weight matrix, and the bias terms can be combined into a single bias.
- The same collapse applies to any number of layers, so a nonlinearity such as ReLU is necessary for stacked layers to gain expressive power beyond a single layer.
- The RT-IoT lab reshapes 81 tabular features into a 9-by-9 grid before applying a convolutional neural network.
- Unlike neighboring pixels in a natural image, adjacent grid positions have no meaningful relationship unless the feature ordering was deliberately designed to encode one.
- A CNN may fit arbitrary training-set patterns or act like a feature-focused classifier rather than learning genuine spatial structure.
- Good validation results in the lab do not establish that the grid represents meaningful spatial data, especially when training and validation samples are similar.
- A historical pneumonia model appeared to find that asthma patients had fewer complications, but the association reflected treatment decisions rather than a protective effect of asthma.
- Clinicians gave asthma patients more intensive or earlier standard-of-care treatment, which contributed to better outcomes in the recorded data.
- Because treatment information was entangled with outcomes, the model learned a misleading association that would not support predictions before treatment decisions.
- This is data leakage: information connected to downstream treatment and outcomes contaminates training signals intended for a pre-treatment prediction.
- The one-dimensional kernel [1, -1] computes the difference between neighboring pixels as its sliding window moves across the input.
- Equal neighboring values produce zero; transitions between 1 and 0 produce nonzero responses, identifying vertical edges.
- The kernel is a finite-difference edge detector, and its positive or negative response depends on the direction of the transition.
- AlexNet was designed for a particular input size, commonly given as 224-by-224 or 227-by-227 pixels, which determines feature-map dimensions throughout the network.
- A different resolution can change the flattened feature count passed to fully connected layers and cause dimension errors in a conventional model.
- LazyLinear may conceal a dimension mismatch during initial setup, but it does not make an existing architecture or its pretrained weights automatically compatible with different inputs.
- Pretrained models such as ResNet-50 are valuable partly because their weights were trained on ImageNet, so preserving expected input dimensions is important when reusing them.
Transcript excerpt
0:00 Yeah, let's go ahead and get started. Welcome back. It's good to see a lot of you here today. As you know, we're jumping around a little bit because this week is our kind of second checkpoint of the semester and by syllabus requirement we must have an exam this week and that's kind of the the biggest thing I wanted us to talk about and spend our time on here today. I did I get a message and sort of forwarded that on that there's an understanding that the testing center is a little bit busier this week than maybe it was back in week five. Can I get a show of hands? How many of you have already signed up for a slot in the testing center? Okay. Okay. Well, those of you who didn't raise your hands, it's fine, but you can look around the room and see what your peers have done and you may want to get get to that and you get to that right now and I won't take it personally. I guess you can sign up pretty early, like pretty early in the semester you could sign up for the whole
0:46 semester nowadays if you want to. So, maybe go sign up for exam three while you're at it. I don't think there's a big advantage of doing it earlier than later. I also don't know what we do if you go to sign up and there's no slots available. So, let's not find out. So, just go sign up for that now if you haven't already. Um otherwise, to the extent there were any surprises on exam one, there should be effectively no surprises on exam two, right? In terms of the topics, I have them listed here, but I hope none of
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Joseph Zambreno.