CMU AI Agents 2026: 14. Lessons from Building OpenHands
Watch on YouTube →
Overview
Graham Neubig traces OpenHands from its 2024 launch as OpenDevin to a flexible agent SDK and a product used for coding workflows and software-development automation. His central design lessons are to provide strong defaults that remain replaceable, favor capable single agents with skills and MCP tools for many coding tasks, and use multi-agent systems selectively for parallel work, independent review, and context management. Repeated automations also create feedback data that can improve agent instructions, as demonstrated by the Skill Refiner research project.
Key takeaways
- OpenHands’ SDK favors strong defaults that developers can replace, combining model choice, tools, conversation state, sandboxing, and safety controls behind one reusable foundation.
- For many coding tasks, a capable single agent with dynamically loaded skills and MCP tools can replace rigid multi-agent workflows while retaining access to specialized instructions and services.
- Multi-agent systems are useful for parallel independent tasks, separate review, and context reduction, but software work adds hard coordination problems such as conflicting edits and integration overhead.
- Agent deployment should match the task: local execution is popular for interactive coding, while cloud workspaces suit background automation such as code review.
- OpenHands automations make issue triage and pull request review repeatable, generating feedback that can be used to improve agent instructions through systems such as Skill Refiner.
- Human-agent interfaces must address more than speed: a study found OpenHands agent users completed tasks in half the time and twice as many tasks as Copilot users, yet users also reported boredom and difficulty verifying results.
Chapters
- OpenHands supports human-in-the-loop coding through a chat interface, with model selection such as DeepSeek and GPT models.
- Users can customize MCP servers, skills, plugins, and apps alongside their model choice.
- A daily workflow automation gathers tasks and reports what Graham Neubig should work on.
- The OpenHands software factory automates recurring repository tasks, including pull request review, issue triage, and Slack monitoring.
- Automations can respond to dependency failures, while the platform aims to remain language-model agnostic.
- OpenHands contrasts with Claude Code and Codex, which are more closely tied to particular model ecosystems.
- Neubig’s code-generation work began in 2016; until around 2023, most tools focused on completion or generating individual functions rather than agents.
- SWE-bench introduced repository-level issues, snapshots, patches, and tests, with early agent results around 5%.
- Cognition’s Devin demo in March 2024 inspired an open-source effort; OpenDevin quickly reached 10,000 GitHub stars and later became OpenHands.
- Claude 3.5 Sonnet began making agents more practical, Claude 3.7 Sonnet improved performance, and 2026 priorities shifted toward deployment, longer runs, parallel tasks, and verification.
- OpenHands replaced its difficult first harness with a second SDK designed around useful defaults that developers can remove or replace.
- The SDK supplies models, tools, state management, workspace options, and customization for models, instructions, and deployment.
- A basic agent can be configured with a model, API key, and tools such as a terminal, file editor, and test tracker.
- The same foundation supports the GUI, CLI, and integrations, with Python APIs intended for direct customization.
- OpenHands represents conversations as tasks, actions, and observations, and custom tools implement an action-to-observation executor.
- Conversation events can be persisted by ID and restored from storage; context condensation summarizes older events while preserving original instructions and recent work.
- Workspaces can be sandboxed by running the agent inside a prepared environment or by routing commands through a context manager.
- Confirmation policies and a security analyzer can restrict which actions an agent may execute autonomously.
- OpenHands’ enterprise users value running coding agents on their own clouds with security, model choice, and customization.
- Neubig recommends the full-featured OpenHands SDK for complex experiments, or mini-SWE-agent when a concise, hackable Python baseline is more useful.
- OpenHands’ Python implementation reflects its machine-learning origins; type checking helps mitigate weaker type safety, while model calls and executed programs usually dominate runtime.
- The SDK can compact context by turn count or token threshold; Neubig notes that current models generally adapt better to different harnesses than earlier models did.
- Neubig argues that improving instruction-following models can perform multi-step workflows from one prompt, reducing the need for fixed multi-agent pipelines.
- OpenHands keeps its core coding tools focused on file editing, command execution, and browsing, while supporting MCP for external services.
- Skills load task-specific prompts when needed, avoiding a single oversized system prompt; OpenHands originally called these components microagents.
- Skills and MCP provide dynamic capabilities within one agent, and Neubig cites multimodal browsing research showing generalist agents can solve tasks that specialized agents target.
- Multi-agent systems can parallelize independent tasks such as handling 10 pull requests, and a separate reviewer may critique work more harshly than the original agent.
- Subagents can return concise results that discard intermediate context, and cheaper models can handle suitable subtasks.
- Software-engineering agents must avoid duplicate or conflicting edits, return the right context, and integrate work reliably; Neubig describes these challenges in research on long tasks such as PaperBench and Commit0.
- Multi-agent approaches can be more straightforward for deep research across separate document sources, while software-evolution benchmarks include SWE-Milestone and ProgramBench.
- Devin began with cloud workspaces, while OpenDevin initially used local Docker isolation; developers later showed a strong preference for local agents on interactive coding tasks.
- Cloud workspaces remain popular for background tasks such as code review, while local agents suit bug fixes and feature implementation.
- Common interfaces include IDEs, agent-focused GUIs, CLIs, and integrations such as Slack or GitHub; Neubig’s audience used CLI most often, with IDEs next.
- A study comparing GitHub Copilot with the OpenHands agent found agent users completed tasks in half the time and finished twice as many tasks, but reported boredom and difficulty verifying results.
- OpenHands automations combine a trigger such as an issue, pull request, or schedule with gathered context, an isolated workspace, an artifact, and human or policy review.
- The issue-triage bot flags unclear reports before implementation or produces acceptance criteria that define when an issue is complete.
- Pull request review uses repository-specific guidelines to assess scope and code quality, addressing low-quality contributions that can result from vague issues.
- Repeated triage and review workflows create reusable examples and feedback, turning recurring development operations into opportunities for system improvement.
- The Skill Refiner project converts delayed feedback from review and revision into edits to agent instructions.
- It represents successful and failed trajectories with evidence, clusters examples using embeddings, and proposes instruction changes based on recurring patterns.
- An evidence gate checks proposed changes, with reruns used to test whether revised instructions produce better reviews.
- Neubig reports that Skill Refiner was cheaper and more effective than comparison methods, and argues that stronger evaluation could support prompt, harness, or model improvements.
- Neubig recommends storing shared context in GitHub or another cloud service and pulling fresh state, with MCP integrations connecting agents to frequently used services.
- Moving work from a local machine to the cloud requires transferring the relevant context; large local datasets may need to remain local or move to globally accessible storage.
- An agent with SSH access to every machine creates a security risk if compromised, and frequent SSH connections can overload a shared login node.
- For cluster work, Neubig prefers running the agent on the cluster and connecting through an SSH tunnel; OpenHands’ agent server can accept connections from outside.
- Multi-agent systems do not automatically reduce token costs: Neubig says one software-engineering experiment cost more but completed tasks more successfully.
- Parallel context management can reduce repeated reading in some settings, but cheaper operation should not be assumed without measurement or an explicit cost objective.
- GitLab and newer GitHub alternatives may offer more agent-native workflows; Neubig sees room to rethink how teams verify completed work without rebuilding all of GitHub.
- Neubig closes by inviting research on agent verification and interfaces, then notes that guest lecturers will take over the remaining course sessions.
Transcript excerpt
0:00 Okay, great. So, I'll I'll start the lecture today. The lecture today is going to be uh quite different than the previous lecture in that I will be focusing a little bit on technical stuff but I'm going to be focusing also a lot on my own experience like building a product in the agentic space that people actually that users actually use and some of the things I've learned there and also maybe a little bit about um how I take the things that I've learned trying to get people to actually use agents and turn them into research projects. So, how how we can think of impactful research projects based on the pain points that people encounter when uh when using something. So, um the first thing I'd like to start out with is just to show people um what OpenHands looks like. I'll talk a little
0:48 bit more about the history, but open hands is uh one of the older open-source uh agents for software development. Um this is our front end. uh we focus on two varieties of uh software development. One variety is just uh kind of like human in the loop uh whatever variety of uh whatever like task you want to do today uh you type it in here in the chat window. The other one is um automation. So if you want to automate something that you do over and over again, you click on one of these automations and set up like code review, a repository man monitor, a slack channel monitor or something like this. So you can say um let's see go with my my favorite model DeepS Flash and say um find my open PRs. What do I
1:40 need to work on today? Find my open PRs on the open hands
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Graham Neubig.