CMU AI Agents 2026: 4. Memory and Skills for Agents
Overview
Daniel Fried explains how agents can preserve useful experience beyond the context window by storing facts, trajectories, and reusable skills, then retrieving or revising that information for later tasks. He compares human-written and automatically induced skills, including Markdown-based Agent Skills, text workflows, and executable code, and surveys methods such as MemGPT, Mem0, Reflexion, Agent Workflow Memory, SkillWeaver, PolySkill, and ReasoningBank.
Key takeaways
- External memory is more portable and inspectable than model-weight updates: a text skill for searching a product can transfer across models and be edited by people.
- Progressive disclosure controls skill-context cost by exposing short YAML metadata first and loading detailed Markdown instructions or scripts only when relevant.
- Skill quality must be evaluated in use: SkillsBench found average gains from curated skills, but irrelevant skills can distract models and lower task performance.
- Executable skills can be tested and composed as tools, reducing repeated environment interactions, but rigid code may fail when a website changes its interface.
- Memory systems need update and deletion operations, as illustrated by Mem0's handling of new, changed, contradicted, or irrelevant facts.
- A large memory is not automatically better: retrieval noise can hurt performance, so agents should select relevant experiences and prune entries that are rarely used.
Chapters
- A shopping agent that learns to search for Sony headphones can reuse the same search process to find a wireless keyboard later.
- Reusable task structure could be stored as raw actions, text instructions, or executable functions.
- A useful skill system must decide what to retain, how to handle conflicting knowledge, and when to edit or delete skills.
- ReAct-style context management retains selected observations and actions, but even million-token windows cannot hold experience across unlimited tasks.
- External files or databases preserve information across tasks and can be inspected or used by different models, unlike model-weight updates.
- Experience can be stored as full episodes, compact facts about a user or environment, or reusable procedures for subtasks.
- A human-written skill can encode a stable policy, such as an organization's code review checklist or deployment procedure.
- Automatically induced skills can be created from agent experience and evaluated through repeated use, but may be noisy or overgeneralized.
- Skills complement memory: memory might record a prior purchase, while a skill describes how to search or complete a purchase.
- The Agent Skills standard uses a required SKILL.md file with YAML name and description metadata, followed by detailed instructions.
- Optional scripts, references, and assets provide executable code, supplementary documentation, and templates.
- Progressive disclosure first exposes concise skill metadata, then loads full instructions and resources only when the model requests them.
- SkillsBench assembled roughly 2 million candidate skills and 400 independently written tasks, with annotators selecting skills relevant to each task.
- The benchmark tests skills across four agent frameworks and about 18 models, comparing task performance with and without skills.
- Curated skills improve average pass rates, but irrelevant or excessive skills can distract models and reduce performance on some tasks.
- SkillsBench results suggest models often create poor skills from a task description alone, so skill induction should draw on actual interactions.
- An OpenHands code review example measures which agent review comments developers incorporate across many pull requests.
- A reasoning model can summarize this feedback and propose skill edits, such as emphasizing repository conventions that developers value.
- MemGPT uses model-directed tool calls to read and write external memory while keeping a smaller working context available to the language model.
- Mem0 extracts candidate facts from new messages, retrieves similar stored facts with embeddings, and uses a language model to decide whether to add, replace, delete, or ignore them.
- Updating and deleting memories matters when facts change, for example when a user changes jobs or reports being laid off.
- Reflexion uses natural-language feedback about a failed attempt to guide the agent's next attempt at the same task.
- Trajectory-memory methods store past observations, actions, tool calls, and sometimes reasoning, then retrieve relevant examples for a new task.
- A common retrieval approach embeds the new task description and compares it with descriptions of stored tasks, optionally filtering by domain.
- A sequence of website actions for searching a product can be reused, but concrete element IDs may change between pages or sites.
- Voyager represents Minecraft behaviors such as crafting and combat as code functions that can call and compose other functions.
- DreamCoder discovers recurring program fragments and abstracts them into reusable functions, offering a model for skill induction.
- In online learning, tasks arrive one at a time and the agent builds a skill library as it encounters recurring subtasks.
- Evaluation asks whether success improves on later tasks as the agent reuses skills learned from earlier tasks.
- A skill lifecycle includes proposing a skill, deciding whether to store it, using it, and pruning, generalizing, or merging it.
- Agent Workflow Memory prompts a language model to identify reusable portions of an agent's web-task tool-call sequence.
- It stores workflows as text descriptions paired with templated actions, such as a generalized procedure for searching different product terms.
- A judge model checks whether the original trajectory appears correct before its induced workflows are added to memory.
- Agent Skill Induction turns trajectory actions into executable functions and rewrites the original trajectory to call those functions.
- Executing generated code before judging its result provides a check of the skill itself; SkillWeaver also uses linters and reward feedback to revise faulty functions.
- Code skills can combine low-level website actions into a few tool calls, reducing repeated page observations and improving both accuracy and interaction efficiency.
- A skill learned on one shopping site can fail on Target if it assumes a dropdown where the site instead uses radio buttons.
- PolySkill separates abstract operations, such as searching or checking out, from website-specific implementations.
- When a concrete implementation does not fit a new site, the agent can create another implementation while keeping only relevant skills active.
- ReasoningBank turns failed experiences into general textual advice, such as using a more specific search query to avoid thousands of irrelevant results.
- Its results indicate that textual strategies can benefit from negative examples, while adding negative examples to stored trajectories or workflows does not provide the same improvement.
- Memory retrieval quality and judge accuracy matter: excessive retrieved experiences can distract models, and usage-based caching can remove seldom-used entries.
- Recent work trains agents with reinforcement learning over sequences of similar tasks while skills are induced, stored, and reused.
- Reward can reflect whether later tasks succeed and whether the agent reuses skills learned from earlier tasks.
- A reuse reward improved success in one reported approach, and future objectives could also penalize oversized skill libraries.
Transcript excerpt
0:01 Okay, welcome back everyone. >> So, um the first assignment is out and uh as Graham said, we we have credits available for you to to work on it for those of you who are enrolled in the class. Do you folks have any logistic questions? Cool. Okay. So we talked in the last lecture about how agents can uh choose what to retain within their context window as a way of you know storing information within a task and using that to um to determine what's important to do better within the task. So a big part of today is going to be about how can agents go beyond the context window and store information across tasks in external stores um that will be useful for them to improve
0:50 performance on tasks in the future. And this is going to be skill induction. Um, and uh, we'll build on some some work that was done in the past on memory systems for agents. Um, and show how to apply those to to agents. But there's also another view on skills. And skills are a way for people to write instructions for an agent that can encode kind of like best practices or the knowledge that you have about how a task should be done. And these are actually going to be complimentary to each other. Um so we'll cover it just a small amount in this lecture but um you can imagine a system that um is sort of having a human in the loop um that's relying on the agent to uh propose some potential best practices from the
1:38 experiences that it has a person edit those and goes back and forth or vice versa like you know having a skill and instruction that's written by a person
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
In this course
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Graham Neubig.