mrkeyoor.com_
Mon 10 Aug 15:24 UTC
LLM Toolsevaluationupdated 10 Aug 2026

open_deep_research

Open Deep Research is a Python and LangGraph agent that searches the web or MCP tools, gathers evidence, and writes a cited report from a question. It gives developers an open, configurable starting point for research automation across several model and search providers instead of locking the workflow to one hosted research product.

Verdict

Open Deep Research is an excellent piece of executable architecture for teams committed to LangGraph, and its provider flexibility and published evaluation path make it more useful than a toy demo. It is not yet a research appliance you should trust by completion status alone. Adopt it as code you will test, instrument, and adapt, especially around provider failures and citation review.

Setup3/5Clear local steps, but several credentials and provider choices remain
Docs4/5Strong overview and evaluation guide, thinner production guidance
Community4/5Large audience with current issue and pull-request activity
Maturity3/5Useful reference code without tagged releases or settled error paths

Who it’s for

  • Python teams that want to inspect and modify a LangGraph-based research agent rather than call a closed research API.
  • Developers comparing model and search-provider combinations with a repeatable research workflow.
  • LangGraph and LangSmith users who want a ready-made graph, Studio interface, traces, and evaluation scripts.
  • Agent builders who need research over custom MCP tools as well as public web search.

Who it’s NOT for

  • Users who need an offline researcher with no paid accounts: the default path calls OpenAI models and Tavily, while the README's full 100-task evaluation can cost about $20 to $100 depending on model choice.
  • Teams whose chosen local or hosted model lacks structured output and tool calling: the README explicitly requires both capabilities.
  • Anyone who needs a polished standalone research application: the quickstart opens LangGraph Studio, and the non-technical Open Agent Platform route is a separate deployment.
  • Workflows that accept a completed run as proof of complete research: current source still catches any child researcher exception and ends that research phase, and an open issue shows the outer graph can then produce a normal-looking partial report.
  • Operators wanting stable release artifacts and upgrade notes: the repository has no GitHub releases, so deployment means pinning a commit or Python dependency state yourself.

Setup reality

The code starts cleanly for someone already comfortable with Python tooling: use Python 3.10 or newer, create a uv environment, sync a sizable dependency set, copy the environment template, and launch the in-memory LangGraph development server with Python 3.11. The missing effort is credentials and fit. The defaults need OpenAI and Tavily keys, LangSmith is part of the documented evaluation path, and alternative models must support tool calls plus structured output. Ollama and OpenRouter guidance lives in issue comments rather than the main quickstart. A production service also needs a user interface or API client, persistent deployment, secrets management, cost limits, observability, and validation of citations and partial-failure behavior.

A research workflow you can actually inspect

Open Deep Research turns a question into a LangGraph run that can clarify the request, form a research brief, delegate searches, compress findings, and write a final report. That is a familiar product experience, but the value here is access to the machinery. Its prompts, state, model assignments, and report generation are inspectable Python.

The current implementation uses different model slots for summarization, research, compression, and final writing. Defaults point to OpenAI, but LangChain's common model initializer opens the door to Anthropic, Google, DeepSeek, Groq, AWS, and other supported providers. Search is similarly replaceable. Tavily is the default, native OpenAI and Anthropic web search are supported, and MCP configuration can connect the agent to private or specialized tools.

This flexibility is the main reason to choose the project. A research team can use a cheaper model to summarize search results, reserve a stronger model for the final report, and direct the agent toward internal sources through MCP. Configuration also controls clarification, concurrent research units, iteration counts, and token limits. You can change the graph when a provider behaves poorly instead of waiting for a hosted service to expose a new setting.

The benchmark is useful, with important limits

The README does more than claim good research quality. It documents an evaluation against Deep Research Bench, a set of 100 English and Chinese tasks across 22 fields. Results include the model assignments, token totals, reported cost where available, commit references, and LangSmith experiment links. The listed GPT-5 run scored 0.4943 on the leaderboard's RACE metric, while the documented default configuration scored 0.4309.

Those figures are evidence about specific configurations, not a universal quality promise. The benchmark uses a Gemini-based judge and expert reference reports, so the score depends on the benchmark's questions, judging prompt, models, and search results. The README warns that running all 100 examples can cost roughly $20 to $100 depending on the selected model, and one listed Claude Sonnet 4 experiment cost substantially more. Reproduction also runs through a LangSmith dataset and export script before submission to the external benchmark.

Still, publishing this trail is better than presenting a few hand-picked screenshots. Teams can rerun the evaluation with their own model mix, compare experiments, and add domain questions. For a legal, medical, or financial research product, that custom set should test source quality, missing evidence, citation alignment, and refusal behavior rather than relying only on one aggregate leaderboard score.

Local setup is for builders, not casual users

The quickstart asks for a clone, a uv virtual environment, dependency sync, and a copied environment file. The package supports Python 3.10 or newer, while the launch command explicitly asks the LangGraph CLI to run with Python 3.11. It then starts a local API on port 2024 and opens LangGraph Studio in the browser. Users enter a question in the message field and change models or tools through assistant configuration.

This is reasonable developer setup, but it is not a complete local application. The defaults require OpenAI and Tavily credentials. The environment template also provides Anthropic, Google, and LangSmith keys. A selected model must support structured output and tool calling; choosing a provider name is not enough if its model or adapter lacks those behaviors. Ollama and OpenRouter instructions are linked from GitHub issue comments, which signals that local-model use needs more assembly and testing than the primary route.

Deployment choices reinforce the project's LangChain orientation. The README points to LangGraph Platform for hosting and Open Agent Platform for a configurable interface aimed at non-technical users. Self-hosters can build another client against the local API, but they must provide authentication, persistence, secret handling, rate and cost controls, monitoring, and a user experience. The repository is a strong agent core, not the whole service around it.

Failure handling is the production warning

Research software has an awkward failure mode: an incomplete answer can look polished. An open issue identifies exactly that risk in the current graph. The child research exception handler contains an unconditional condition that treats any exception like the end of the research phase. The outer graph then proceeds to final report generation. A provider outage, MCP failure, network error, or faulty child run can therefore yield an apparently successful report based on partial notes.

A proposed fix is open, but the current main-branch source still contains the behavior. Production callers should not equate graph completion with evidence completeness. They need explicit failure events, source-count checks, citation validation, and traces that reveal which delegated branches returned usable material. Human review remains necessary for consequential work.

MCP setup has its own paper cut. A long-running issue shows that passing MCP_CONFIG as an environment JSON string was not parsed into the expected model, leaving Studio configuration as the working route. An open pull request now targets environment parsing, but this is another reason to test the exact commit and deployment method rather than infer support from the feature list.

Active code without a release contract

The repository was pushed on August 8, 2026, and issues and pull requests were updated in August. The GitHub open count combines both, with much of the recent queue consisting of proposed fixes and dependency work. Activity is clearly alive. What is missing is a GitHub release history: the latest-release endpoint returns no release, and the package file still identifies version 0.0.16.

Documentation is strong on architecture, configuration categories, benchmarking, and ways to run the graph. It is thinner on self-hosted production operations and concrete provider matrices. The legacy directory also contains older workflow and multi-agent implementations, useful for study but explicitly described as less performant than the current graph.

Use Open Deep Research when you want a transparent, adaptable LangGraph research core and are prepared to own quality control. If you want a finished search product or a stable API with release guarantees, choose a more packaged alternative. Here, openness buys control, but it also hands you the testing burden.

Alternatives

ProjectWhat it isPick it when
GPT ResearcherAn autonomous research agent with a broader application and integration surface.pick this instead when you want a more product-shaped researcher with established application interfaces, not primarily a LangGraph reference implementation.
STORMA research system focused on building long-form, Wikipedia-style reports with citations.pick this instead when structured knowledge curation and article generation matter more than swapping among many agent tools.
VaneA self-hostable AI answering engine built around web search and cited responses.pick this instead when you want an end-user search interface rather than a research graph to embed and modify.

What people are saying

  1. [github-trending] langchain-ai/open_deep_research

Sources

  1. Open Deep Research README
  2. Deep Research Bench leaderboard
  3. Child researcher exception issue
  4. MCP environment configuration issue
  5. Open Deep Research activity