CMU AI Agents 2026: 10. Deep Research

Overview

Deep research agents answer complex questions by planning searches, inspecting evidence, revising queries, and synthesizing cited responses; building them well requires progress in evaluation, training, and retrieval together. The lecture surveys benchmarks such as BrowseComp, BrowseComp-Plus, and FinSearchComp; explains rubric-based evaluation and reinforcement learning for open-ended answers; and shows how reasoning-aware retrieval can improve agent accuracy while reducing search calls.

Key takeaways

Deep research requires iterative planning and evidence gathering: unlike single-page RAG, an agent must inspect search results, identify missing evidence, and revise its queries before synthesizing citations.
BrowseComp-style tasks can require long contexts and many tool calls; reported ablations found near-zero performance with an 8,000-token context versus much stronger results as context grew toward 128,000 tokens.
Reference-answer similarity is a weak evaluator for open-ended research: conventional metrics predicted expert preferences only about 57–62% of the time, while high-quality rubric judgments achieved roughly 79–80% agreement with experts.
For open-ended agent training, search-grounded evolving rubrics provide a more useful reward than random feedback; a strong supervised fine-tuning starting point also materially improves subsequent reinforcement learning.
Retrieval should use an agent’s reasoning state, not only its latest search query: Agent-IO’s 4B reasoning-aware embedder improved accuracy and reduced search calls, while full-history inputs could hurt due to excess context.
Evaluation remains a core research bottleneck because benchmarks may be synthetic, rubric scores may be biased, and agents can retrieve benchmark files directly; future tests should also measure whether research outputs improve downstream human decisions.

Chapters

0:00 Deep Search Agents Move Beyond Single-Page RAG

A question like finding a CMU office number can be answered with one search and one page, making a simple retrieval-augmented generation pipeline sufficient.
Questions such as whether AI agents synthesize scientific literature as well as human experts require evidence from multiple papers and iterative searches.
A deep search agent plans what to learn, calls external search tools, inspects results, revises its plan, and produces a cited synthesis.

4:03 Why Natural Questions Does Not Measure Deep Research

Natural Questions was built from real Google queries with answers grounded in Wikipedia, so many examples are short, general-domain lookups.
Modern deep research tasks often span expert or long-tail domains and cannot be answered from a single web page.
Short entity answers do not test long-form synthesis, and an answer without supporting evidence makes its correctness difficult to assess.

6:35 BrowseComp and BrowseComp-Plus Test Multi-Step Search

OpenAI’s BrowseComp contains more than 1,000 human-written questions constructed backward from known answers by combining multiple clues.
The benchmark validates questions through model-assisted checks and human review; its questions require satisfying several conditions rather than recalling one fact.
Early frontier models with web search and human experts struggled on BrowseComp, while task-specific training substantially improved model results.
Reported ablations show that complex tasks benefit from long contexts and many tool calls: performance rose sharply as context increased from 8,000 toward 128,000 tokens, with up to 128 calls tested.

12:13 Expert Benchmarks for Finance, Medicine, and Research

FinSearchComp adapts multi-step search evaluation to finance, using domain experts to create and verify questions grounded in financial sources.
Some finance questions require both specialized knowledge and calculator tools to produce exact numeric answers.
Other focused benchmarks include medical search tasks and ResearchBench, which tests whether an agent can identify a paper meeting multiple constraints.
Expert involvement in both question creation and answer verification is important for credible domain-specific benchmarks.

14:12 Rubric Evaluation for Open-Ended Research Answers

Long-form research answers can have many valid formulations, so comparing them with one or two reference answers misses acceptable alternatives.
Lexical-overlap metrics can reward a response that repeats reference wording while making a consequential factual error; expert-preference studies found conventional metrics only about 57–62% accurate.
OpenScholar uses expert-written rubrics: concrete criteria describe what a strong answer should contain, and a language model judges each criterion before scores are combined.
Expert and model judgments reached roughly 80% and 79% agreement, respectively, when the rubrics were high quality, but creating a rubric could take a PhD expert nearly an hour.

20:08 Citation Checks, Synthetic Rubrics, and Benchmark Choices

ResearchQA generated about 21,000 questions and 120,000 rubrics from survey papers, but expert audits found errors in roughly 15% of model-generated rubrics.
Rubric errors included nonexistent papers, repeated questions, and criteria too vague to distinguish strong answers from weak ones.
DeepResearch Bench applies FACT-style checking by matching answer claims to cited URLs; one supported claim out of two yields 50% citation accuracy.
For scoped projects, BrowseComp suits verifiable short answers, BrowseComp-Plus reduces live-search costs with a fixed corpus, and DeepResearch Bench 2 is a candidate for open-ended evaluation.

24:09 The Search–Inspect–Revise Agent Loop

An agent starts with a plan, issues searches through tools such as web search, browsing, or paper-search APIs, and adds retrieved documents to its context.
It then evaluates what the evidence establishes, decides what remains unknown, and searches again before producing a response with citations.
The loop can require many tool calls, so agent behavior and retrieval quality both affect whether the final synthesis is useful.

25:28 Creating Training Questions from Knowledge Graphs

A scalable task-generation method samples connected facts from knowledge graphs such as Wikidata and turns them into multi-condition questions.
For example, linked facts about Allen Newell, CMU, awards, and coauthorship can be combined into a question whose target entity is the gold answer.
This construction resembles BrowseComp’s backward design and provides complex question-answer pairs for training, though transfer to other deep research tasks remains uncertain.
Stronger teacher models generate agent trajectories; successful traces can be selected using answer matching or format rewards for supervised fine-tuning.

30:26 Agentic Mid-Training, SFT, and GRPO

Agentic mid-training mixes tool-using trajectories into continued pretraining before supervised fine-tuning; with the same SFT data, a mid-trained model performed better on complex BrowseComp-style tasks.
For verifiable short answers, Search-R1 applies reinforcement learning with a correctness reward: the policy generates multiple attempts and learns from which trajectories reach the answer.
GRPO compares responses within a group, raising the relative advantage of correct attempts and lowering that of failures, with clipped updates and a KL penalty to limit policy drift.
Because web tools make training slow and costly, systems use asynchronous rollouts, cached results, robust APIs, and local search indexes.

34:00 Evolving Rubrics Provide Rewards for Open-Ended Research

Long-form research quality depends on coverage, relevance, factuality, and citations, so exact-match rewards designed for short answers are inadequate.
A rubric-based reinforcement-learning approach starts with search-grounded rubrics and generates new positive and negative criteria by contrasting policy outputs.
Positive rubrics reward desirable behavior, while negative rubrics target specific errors; a buffer retains useful criteria as the policy improves.
In the described setup, a language model compared eight trajectories per question to generate rubric feedback, while rubric importance weighting and trajectory selection remained open design questions.

36:16 Rubric-RL Results and the Value of a Strong SFT Start

On an open-ended research benchmark, the rubric-trained model achieved strong results with roughly 1,000 times fewer inference resources than the compared proprietary deep-search systems.
Starting reinforcement learning from a base model was weaker than starting from even a small SFT run; using 5% of the SFT data improved results by about ten percentage points after 600 RL steps.
Random-reward training did not improve performance, whereas search-grounded rubrics did; rubrics generated without external evidence were less helpful.
A stronger rubric generator such as GPT-4.1 outperformed a Qwen base model in the reported ablation, but the latter still delivered gains at lower cost.

45:00 Search Foundations and Reasoning-Aware Retrieval

A deep research agent is limited by its search foundation: local BM25 or embedding indexes work for fixed corpora, while broader tasks may need web or scholarly APIs.
Standard embedding retrieval encodes documents offline and ranks them against a query embedding using nearest-neighbor search.
Contrastive training brings a query closer to its relevant document than to negatives; negatives can include other examples in the batch or hard negatives mined by BM25.
Agent-IO improves retrieval by conditioning on the current reasoning as well as the query, allowing search to reflect the task and findings from earlier steps.

47:25 Agent-IO Ablations and the Case for Concise Search Context

On BrowseComp-Plus, replacing BM25 with a reasoning-aware embedding model improved answer accuracy while reducing the number of search calls.
The reported Agent-IO model is 4B parameters, smaller than the compared 8B embedding model, and its training improved performance across different agents and benchmarks.
Adding only the latest reasoning to a query gave limited gains without training; reasoning-aware training produced stronger improvements.
Feeding the full trajectory could hurt retrieval because it made the input too long; the reasoning immediately before the current query was most useful because it summarized intent and prior findings.

55:47 Evaluation Gaps, Model Limits, and Research Directions

The lecture identifies benchmark quality as a bottleneck: rubric evaluators can favor longer answers, while BrowseComp questions may be synthetic rather than representative of real user needs.
Frontier models retain advantages in long-context management, planning, and handling many searches; smaller models can be competitive on selected synthesis tasks but do not dominate every benchmark.
Benchmark contamination is difficult to rule out, and agents may retrieve benchmark files directly from sites such as Hugging Face unless those sources are blocked.
A further research gap is evaluating whether a research report helps people complete downstream tasks, rather than scoring only the final response.

58:20 Q&A: Project Costs, Model Baselines, and Evaluation Risks

For student projects, BrowseComp-Plus offers a controlled corpus and lower inference cost; live web-search training can cost thousands of dollars, with one run estimated at $3,000–$4,000.
Qwen-family models and other strong open checkpoints are reasonable starting baselines; local BM25 or embedding indexes avoid repeated live API calls.
A specialized small model may beat a larger general model on a carefully chosen task, but broad claims against frontier systems are difficult because model strengths vary by benchmark.
Rubric-based reinforcement learning may extend to other open-ended tasks, but reliable evaluation and safeguards against retrieving original benchmark data remain essential.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Graham Neubig.