Save this video free

In-context learning and instruction tuning

Mohit Iyyer · 1:10:25 · Watch on YouTube

Part of University of Maryland CMSC 723. See all 11 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Mohit Iyyer explains how pre-training gives a language model broad knowledge but does not reliably teach it to follow user intent, then contrasts prompt-based in-context learning with post-training that changes model parameters. Using an OLMo 3 7B base-model demo, he introduces GPT-3-style prompting, supervised fine-tuning and instruction tuning, catastrophic forgetting, reinforcement learning with human or verifiable rewards, and distillation, emphasizing that pre-training and scale support generalization to unseen tasks.

Key takeaways

Chapters

0:00 OLMo 3 7B Demo: What a Pre-Training-Only Base Model Can Do
7:00 Why Web-Text Pre-Training Does Not Guarantee Instruction Following
11:00 GPT-3 Prompting: From Zero-Shot to Few-Shot Examples
17:00 Chain-of-Thought Prompts and Web Data Contamination
22:00 In-Context Learning, Sampling Variability, and Base-Model Access
31:00 Two Routes to Better Instruction Following: Prompting and Post-Training
37:00 From a Pre-Trained Base Model to an Instruction-Tuned Model
43:00 Reinforcement Learning: Optimizing Responses by Reward
47:00 Teacher-Model Distillation and Online Versus Offline RL
52:00 SFT Loss Masking and the Risk of Catastrophic Forgetting
57:00 Instruction Tuning Uses Diverse Tasks to Teach a Meta-Task
1:00:00 Why Pre-Training Enables Generalization to Unseen Instructions
1:07:00 Scaling Improves Instruction Generalization, and What Comes Next

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:00 Okay, let's get started everyone. So, today we are going to be covering or starting the post training half of the class. So, we've talked about pre-training up till the first exam and the rest of this class will focus on post training as well as agents and applications of language models. Um All right. Let's get started. So, first I wanted to cover some of the kind of motivations behind this pre and post training split and why it is that model developers started doing post training in the first place.

0:50 So, I made this little well, I should say Claude made this little demo that serves a base model. So, we can see what this is. This is the Olmo 3 7 billion parameter base model. It was released When was it released? November Sorry, October of 2025. So, it's about 1 year old at this point. We don't really get many of these open base models released anymore. So, this is probably one of the most recent ones that that we have. Base here means that this model has just undergone pre-training. So, there's no supervised fine-tuning or reinforcement

1:39 learning, which are all things that we will cover in the upcoming lectures. So, let's just take a look at some of the things that now everyone here is equipped to hopefully understand. We see here that the model was trained on 6 trillion tokens, which seems large. It definitely is large. However, the frontier models are pre-trained on way more than that. It's got 32 layers. So, remember here each layer is like a transformer block. So, we have the self-attention, the multiple heads, and then the feedforward layer. We've got different heads. So, here 32 heads and

2:28 the context length is the length of the sequence that goes into

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Mohit Iyyer.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.