Save this video free

1786 lecture3 Sep22 26

Jonathan Rose · 1:41:00 · Watch on YouTube

Part of University of Toronto ECE 1786. See all 10 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Jonathan Rose connects ECE 1786's word-embedding assignment to large language models: both train neural networks with cross-entropy and backpropagation, but LLMs predict the next token from long sequences and generate text through an autoregressive loop. He argues that scale helped this approach generalize to higher-level concepts, then applies the idea to agentic systems, where LLMs work with software through a harness, and to prompt engineering, where small example sets and held-out testing can validate behavior without training new model parameters.

Key takeaways

Chapters

0:00 Assignment One, AI Assistance, and Course Logistics
4:00 Why Assignment One's Embeddings Matter for LLMs
4:45 The Actual Inputs and Outputs of the Embedding Model
7:00 LLMs Predict the Next Token from Many Embeddings
13:00 Turning Next-Token Probabilities into Generated Text
22:00 The Autoregressive Loop and Repeated Inference
32:00 Why Large Language Models Improved
36:00 Ilya Sutskever's Bigger-Is-Better Bet
43:00 Next-Word Prediction as a Test of Broad Knowledge
53:00 Conceptual Generalization and the Random-Parrot Counterargument
58:32 Base Models, Instruction Tuning, and Safety Constraints
1:00:30 LLMs as a New Engine for Creative Applications
1:05:00 Agentic Systems Combine LLMs and Regular Software
1:09:00 Tool Calls, the Harness, and Orchestrating LLMs
1:13:00 Parametric, Contextual, and Harness Knowledge
1:18:00 Prompt Engineering for Questions and Data Processing
1:22:00 When to Ask an LLM to Help Write a Prompt
1:25:00 Classification and Reflective Listening Prompts
1:31:00 A Prompt-Validation Workflow Using Examples
1:37:00 Why Rose Prefers Prompting Before Fine-Tuning

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:03 Good morning everyone. Welcome to ECE 1786 lecture number three. Question for you. You all look like you're waking up just like me. How did assignment one go? How did it go for you? >> Going fine. >> Going fine. Okay. >> Yeah. Oh, so you are finished. Okay, good. Yeah, it It was a lot of work like I warned you. What was the hardest part for assignment one? Oh, that's

0:51 that's painful. Yeah, you didn't realize you have to produce text. Every assignment you produce text and pop it in so you make it easier. So, what was the hardest part for you? >> AI was >> Yeah. Yeah. I guess that's hard. Did Did you learn more or is that a silly thing to do to make for me to make you do my brain? I don't It's only one. >> Yeah. So, you said you don't think you'll do it in the future. Yeah. I don't think many of us will. My worry was that by using the AI you would it would figure out everything for you. The syntax I don't care about. Right. It was it's the understanding. Yeah, Michael. >> So I I cheated.

1:40 Um but you're right. It very much did exactly what you said. >> Yeah. It's just going to run ahead and give you the answer. [laughter] >> Not looking. >> Yeah. Yeah. So uh it we will be looking for evidence that you cheated. uh uh I know you're taking the course not for credit but uh uh I want to encourage you to learn not to let let it to do do your thinking for you. I mean it's the subject of this course. So uh and remember I'm going to let you do it on on both sides. Uh when we get to the

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Jonathan Rose.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.