Save this video free

CMU AI Agents 2026: 6. Agents for Coding and Software Development

Graham Neubig · 1:18:49 · Watch on YouTube

Part of Carnegie Mellon University 11-768. See all 14 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Graham Neubig explains how coding models and software-development agents are built, from code-aware pretraining, infilling, and long-context training to tool use, evaluation, and reinforcement learning. He argues that reliable agents depend not only on model capability but also on high-quality tests, effective file-editing interfaces, realistic training environments, and support for the broader software-development lifecycle.

Key takeaways

Chapters

0:00 From Code Generation to Software-Development Agents
3:02 Three Capabilities Behind Single-Step Code Models
7:42 Pretraining Code Models on Clean, Diverse Data
12:49 Code Tokenization and Programming-Language Biases
19:18 Mid-Training for Code and Long Repository Context
22:57 Fill-in-the-Middle Training for Code Editing
28:08 Learning Code Edits from Commits, Tests, and Execution
30:47 Evaluating Code Similarity and Program Behavior
38:04 Code Benchmarks, Pass@k, and Quality Beyond Correctness
41:00 Reinforcement Learning from Test Results
47:19 Repository Agents: Localize, Edit, and Verify
51:27 Bash Tools and Reliable File-Editing Interfaces
58:11 Code Localization and the SWE-bench Evaluation Pipeline
1:02:01 Scaling Coding-Agent Training Data
1:08:20 Multi-Harness Training and Front-End Agent Evaluation
1:12:37 Beyond Issue Fixes: The Full Software-Development Lifecycle

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:00 Okay. Um, today I'll I'll be talking about agents for coding and software development. So, presumably many or mo most or all uh people here are using coding agents. So, um I I barely need to describe what their underlying functionality is. Um but I would like to talk about um how we can build coding agents and what the necessary things we need to have in order to build coding agents are. And um actually I just realized that the title of this slide is uh maybe uh a little bit of a misnomer. The first thing that I'd like to talk about is what it means to have a language model. Not a not an agent, but a language model that's able to write code. Um, and so

0:49 there's different uh things you can do. The kind of obvious thing is you say in English um I want you to write a program that uh visualizes this data for me. Um and you give it like a CSV uh file in your prompt and uh creates a bar chart uh for this data. Um, how many people do this sort of query like frequently? Doesn't necessarily need to be a bar chart, but like write write this program for me to do this task. Probably a lot of people uh and you write it in like chat GPT or a coding agent or something like that. Um, however, there's a lot of other tasks that involve generating code that's not just like a oneoff like prompt, but you

1:40 can also do um you can uh have it modify repositories. So, you can have it modify like multiple files. Um, and so this would be something more like in this repository uh fix this issue or make this improvement or something like that.

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Graham Neubig.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.