Save this video free

Pretraining vs post-training

Mohit Iyyer · 1:10:56 · Watch on YouTube

Part of University of Maryland CMSC 723. See all 11 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Mohit Iyyer compares transformer pretraining, which uses trillions of tokens and next-token prediction to build linguistic and world knowledge, with post-training, which shapes an existing model to follow instructions using techniques such as supervised fine-tuning. He explains how attention masks and loss masks distinguish these objectives, then contrasts causal language models with BERT-style encoders and encoder-decoder transformers, before connecting quadratic attention cost to more efficient mechanisms.

Key takeaways

Chapters

0:00 From Random Transformers to Pretrained and Chat Models
6:00 Pretraining Data: Common Crawl, Books, and Filtering
11:00 Pretraining Objectives, Model Scale, and Emergent Capabilities
14:00 Post-Training: Instruction Following, Safety, and Uncertain Scaling
21:00 Supervised Fine-Tuning and Synthetic Training Data
24:00 Why Pretraining and SFT Share the Same Core Objective
26:00 Attention Masks and Loss Masks in Pretraining
29:00 Chat Templates and Assistant-Only SFT Loss
36:00 SFT Batching, Prompt Loss, and Training Tradeoffs
41:00 BERT's Bidirectional Attention and Masked-Token Objective
46:00 Why BERT Is Not a Standard Autoregressive Generator
53:00 Encoder Models for Retrieval, Classification, and Diffusion
56:00 The Original Transformer: Encoder-Decoder Translation
1:05:00 Quadratic Self-Attention and Efficient Attention Methods

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:01 Okay, let's get started. So today we are going to um continue with our discussion of transformers but uh start talking about the different ways in which they can be applied. And so we're going to spend quite a lot of time today talking about different uh attention masks and the implications of each of these masks on what the model can and cannot model and what the um end applications of these models um are. So um just as a quick review of where we are at this point, we have um kind of finished covering the core architectural basics of a transformer. You now know the self

0:51 attention layer, the multi head aspect, the masking, um the feed forward layer that gets split into several different experts in ane. Um, so we're basically pretty up to date with the state-of-the-art in um, architectures for language models. Obviously, there's a bunch of low-level tweaks here and there, but the the general concept is um, it should be clear. Uh, so now we're going to move on to the actual training of these models and scaling them. So the theme for this class is and actually for quite a few of the lectures that follow is pre-training versus

1:38 postraining. Um and at this point it would be worthwhile to um kind of draw a diagram of the pipeline in which um the big labs train their language models like chat GBT or a cloud. Um what we've talked about so far is this process where we start with a random a randomly initialized transformer So again, this means that all of the parameters in this model, the embeddings, the WKD for all of the heads and layers, the feed forward weights for all of the

2:26 experts at all of the layers. All of

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Mohit Iyyer.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.