Save this video free

CMU AI Agents 2026: 13. Agentic Safety

Graham Neubig · 54:11 · Watch on YouTube

Part of Carnegie Mellon University 11-768. See all 14 lectures and search them

These notes were made with YouTube Collector. Paste any YouTube link and get the same: a summary, chapters and a searchable transcript.

Overview

Graham Neubig organizes agentic safety around concrete, current threats: malicious users, prompt injection, agent errors, and improper means, illustrated by deleted inboxes, production databases, leaked documents, and compromised software releases. He explains practical defenses including red teaming and safety training, sandbox isolation, least-privilege and short-lived credentials, runtime monitoring, and benchmarks such as Open Agent Safety, emphasizing that useful agents require balancing capability with controlled access.

Key takeaways

Chapters

0:00 Agent Failures: Deleted Conversations, Production Databases, and Data Leaks
7:16 Four Agent Threat Models: Malicious Users, Prompt Injection, Errors, and Improper Means
12:32 Pragmatic Safety: Balance Agent Permissions Against Irreversible Harm
16:53 Human and Automated Red Teaming for Agent Attacks
21:03 Supervised Safety Training and Reinforcement Learning Against Attacks
24:26 Sandbox Design for Agent Evaluation and Training
37:03 Docker, Apptainer, Cloud Sandboxes, and Shared-State Risks
40:24 Least-Privilege Credentials and Credential Brokers
46:01 Tool-Call and Chain-of-Thought Monitoring for Unsafe Actions
52:57 Runtime Tool Guards and Agent Safety Benchmarks

Keep these chapters and the full searchable transcript in your own library.

Transcript excerpt

0:00 So, unfortunately, we only have a short time to talk about this very important topic of agent safety. Uh, one one second for everybody to quiet down a little bit. Thanks. Okay, cool. So agent uh agents are becoming very high stakes uh because they have been basically rolled out to the entire world very quickly uh before uh we have you know fully been able to think through all of the safety issues and this is not a theoretical problem but uh it's uh like actually causing uh causing real issues. So this is an example that I think we

0:49 also had at the very beginning of the class. Uh which is where an openclaw user basically asked uh their agent to wait for approval before changing their inbox. And um it got a uh said stop, don't do anything. And then it said uh get all remaining old stuff and nuke it. And uh then in a later exchange, I asked you to not uh action on anything until I approved. Do you remember that? Uh yes, I remember and I violated it. You are right to be upset. Um so I I actually have a rather embarrassing story that my agent just did something to me today [laughter] that was not uh not quite this bad, but it deleted all my previous conversations uh on my on my computer. So I lost all

1:37 of my previous conversations with the agent due to it. Similarly, uh I basically asked it to fix uh fix my agent so it stopped crashing and it was like all the old conversations are causing the agent to crash so I'll just go delete them. So uh that was uh disappointing for me but this is something that's not uh not very uh not just theoretical.

This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.

In this course

Related notes

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Graham Neubig.

Want the full transcript?

Save this video in YouTube Collector to get its complete searchable transcript, your own AI summaries, and a library that keeps every video you collect in one place.