mrkeyoor.com_
Tue 08 Sept 16:07 UTC
LLM Toolsevaluationupdated 08 Sept 2026

openai-agents-python review

The OpenAI Agents SDK is a Python framework for running language-model agents with tools, handoffs, guardrails, session memory, human approvals, and tracing. It gives developers one runner for text, long-running sandbox work, realtime sessions, and voice pipelines instead of making each application build those control loops itself.

Verdict

Our install took 15 seconds, but pytest reported 135 failures and 65 collection/setup errors, so the OpenAI Agents SDK is easy to try and hard to approve from the README path alone. Use it for a Python agent product that needs OpenAI Responses, tools, handoffs, approvals, and tracing under one runner. Before production, test the exact extras you use, decide where traces may go, and verify each non-OpenAI model or sandbox path separately.

We ran it

Lab card: what happened when we ran openai-agents-pythonScreenshot of openai-agents-python (openai.github.io/openai-agents-python)
Install✓ · 15s67 packages · 82 MB
Build✓ · 4s
Tests✗ · 51s98 passed · 135 failed · 6 skipped · 65 errors of 298 (pytest)
Known vulns0(pip-audit)
Repo1582 files~438,313 lines of source · 24.7 MB · 5 CI workflows · tests dir

Answers from our run

Does openai-agents-python build from source?

Dependencies installed in 15 seconds (67 packages), and the build succeeded in 4 seconds. We cloned commit 544b8b0 into a clean Debian container with 3 CPUs and no project-specific setup.

Do openai-agents-python's tests pass?

Not all of them: 98 of 298 passed and 135 failed when we ran the project's own test command (pytest), with 65 collection errors. Some failures need services or credentials a bare container does not have.

Does openai-agents-python have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use openai-agents-python?

Teams whose release gate requires a clean full suite from the checked-out repository: our run had 135 failures and 65 collection/setup errors.

What are the alternatives to openai-agents-python?

LangGraph, Pydantic AI, AutoGen. Our install took 15 seconds, but pytest reported 135 failures and 65 collection/setup errors, so the OpenAI Agents SDK is easy to try and hard to approve from the README path alone.

Setup3/515-second install; full suite hit 135 failures and 65 errors
Docs5/5The official guides cover runners, providers, traces, and sandboxes
Community5/529,268 stars and version 0.22.1 shipped on September 8, 2026
Maturity3/5Wide feature set and fast releases, but our suite failed heavily

Who it’s for

Python teams building tool-using agents on the OpenAI Responses API and wanting an official orchestration layer.
Developers who need handoffs, agents-as-tools, approval pauses, session storage, and run tracing in one package.
Voice or realtime builders willing to install the relevant extras and test the event flow end to end.
Mixed-provider teams prepared to check feature differences for every model adapter they select.

Who it’s NOT for

Teams whose release gate requires a clean full suite from the checked-out repository: our run had 135 failures and 65 collection/setup errors.
Zero Data Retention organizations that require the built-in OpenAI traces dashboard: the tracing guide says tracing is unavailable under ZDR.
Applications that cannot risk prompt or tool data entering traces without a configuration review: tracing and sensitive generation and function data capture are enabled by default.
Windows developers expecting the Unix-local sandbox path: the README directs Windows users to Docker or a hosted client, and issue 4852 reports symlink failures for ordinary non-elevated Windows users.
Teams expecting identical behavior across the README's 100-plus model claim: the models guide tells mixed-provider users to review feature differences and describes third-party adapters as beta.

Setup reality

Our sandbox install succeeded in 15 seconds, adding 67 packages and using 82 MB. The build passed in 4 seconds. Pytest failed after 51 seconds: 98 passed, 135 failed, 6 skipped, and 65 collection/setup errors were reported of 298. Pip-audit found 0 known vulnerabilities.

The default OpenAI path needs OPENAI_API_KEY; other providers need their own credentials and adapter configuration. Redis, MongoDB, SQLAlchemy, Dapr, voice, Docker, and hosted sandboxes use separate extras or services. Tracing is enabled by default and can send generation and function inputs and outputs unless redacted or disabled.

Python 3.10 or newer is required. Unix-local sandboxing runs on macOS and Linux; Windows needs Docker or a hosted sandbox for the documented path. The failing log repeatedly says async functions were not natively supported, including encryption and MongoDB session tests, but it does not prove why support was unavailable in our container.

Four runner styles share the Agent and Runner core

The README documents 4 primary ways to execute work: a text agent, a sandbox agent, a realtime agent, and a voice pipeline. The ordinary Agent combines instructions, a model, tools, guardrails, and optional handoffs. Runner drives the model loop until it gets a final result or reaches a stopping condition. This is enough structure to avoid rewriting tool-call dispatch and handoff plumbing, while leaving application prompts, tool permissions, and business policy in the developer's code.

The feature surface extends well past a small loop. Sessions persist conversation history through local or external stores, human-in-the-loop support can pause a run for tool approval, and agents can call other agents as tools or transfer control through handoffs. Realtime and voice add separate event and audio paths. Sandbox agents preserve workspaces across longer jobs and can run commands or patches. Each layer is optional, but a production system must test the exact combination rather than infer behavior from the 1-page quickstart.

The 100-plus model claim comes with feature differences

OpenAI's README says the SDK supports the Responses and Chat Completions APIs plus more than 100 other language models. The model guide is more careful: OpenAI-only applications are directed to the Responses path, while mixed-provider users are told to review feature differences. LiteLLM and any-llm are optional adapters, and their setup paths are marked beta. A common Agent interface does not make every provider support hosted tools, reasoning controls, usage fields, or streaming in the same way.

The base package requires Python 3.10 and depends on the OpenAI client, Pydantic, MCP, Requests, and WebSockets. Provider adapters arrive through separate extras, with any-llm requiring Python 3.11 or newer. This modular packaging keeps a basic app smaller, but example code can quietly cross into another dependency group when it adds Redis sessions, MongoDB, encryption, voice, or a sandbox provider. Pin the extras and run a representative workflow against the actual model endpoint.

What happened when we ran it

Our sandbox installed commit 544b8b0 in 15 seconds, pulling 67 packages and using 82 MB on disk. The build completed in 4 seconds. We used a clean unprivileged Debian container with 3 CPUs, 8 GB of RAM, no secrets, and Python 3.12. The checkout was much larger than the installed base suggests: 1,582 files, about 438,313 lines of source, and 24.7 MB before dependencies.

Pytest ended with exit code 1 after 51 seconds. It reported 98 passed, 135 failed, 6 skipped, 65 collection/setup errors of 298, and 3,036 warnings. The log tail repeatedly says async functions are not natively supported, including encryption and MongoDB session cases. The project's dev dependency group lists pytest-asyncio, but the supplied log does not establish why async support was unavailable. Pip-audit found 0 known vulnerabilities. The repository had 5 CI workflows, a tests directory, and no Dockerfile.

Tracing sends sensitive span data by default

Tracing wraps runner calls, model generations, function tools, handoffs, guardrails, and audio operations. The default processor exports batches to OpenAI's trace backend, which gives developers one timeline for a run. The privacy setting deserves attention before the first real prompt: trace_include_sensitive_data defaults to true, so generation spans may store model input and output, while function spans may store tool arguments and results. Audio spans can include base64-encoded PCM data under their own setting.

You can turn tracing off globally with OPENAI_AGENTS_DISABLE_TRACING=1, disable it for one run, replace the processors, or keep spans while excluding sensitive content. The guide also says tracing is unavailable for organizations using OpenAI APIs under a Zero Data Retention policy. A custom processor can send data elsewhere, but that becomes another observability component to secure. Treat trace configuration as part of the data-flow review, especially when tools touch customer records or internal systems.

Windows sandbox work needs Docker or a hosted client

UnixLocalSandboxClient is the simplest local option on macOS and Linux. It runs commands as host processes, retains local filesystem and network access, and inherits the complete host environment unless configured otherwise. Filtering environment variables reduces accidental credential exposure, but it does not create process isolation. Docker provides a stronger local boundary and can disable networking with network_mode="none". Hosted clients add provider accounts, credentials, storage rules, and their own operational costs.

Windows is outside the documented Unix-local path. The README tells Windows users to install the Docker extra or choose a hosted sandbox. Open issue 4852 adds a narrower warning: on a non-elevated Windows 11 machine without Developer Mode, 12 sandbox tests failed because creating symlinks raised WinError 1314. GitHub's Windows runner can create those links, according to the report, so its green status does not reproduce the ordinary-user restriction. Test archive extraction and workspace paths on the deployment class you support.

Version 0.22.1 shipped on September 8, 2026

GitHub listed 29,268 stars, 14 open issues, and 23 open pull requests when fetched. The latest push was September 8, 2026, and v0.22.1 was published the same day. Its release notes span core runs, MCP, sandboxes, sessions, tracing, realtime, and voice. They also include fixes for approval resume ownership, concurrent session writes, Docker cleanup, sensitive trace redaction, and strict handling of tool arguments. This is active maintenance across security-sensitive control paths.

That pace is useful, though version 0.22.1 still asks buyers to pin and test upgrades. Our 15-second install makes a prototype cheap, while the 135 failures and 65 setup or collection errors block a clean bill of health for commit 544b8b0 in the stated sandbox. Adopt the SDK when its runner matches the system you need, then narrow the provider, session backend, tracing policy, and sandbox implementation. The abstraction is only as trustworthy as those concrete choices.

Alternatives

ProjectWhat it isPick it when
LangGraph gh↗A graph-based runtime for stateful agent workflows and explicit control flow.pick this instead when durable graph state and visible branching matter more than an OpenAI-first runner.
Pydantic AIA typed Python agent framework from the Pydantic team with broad model support.pick this instead when typed outputs and provider choice are the main design priorities.
AutoGen gh↗Microsoft's framework for event-driven single-agent and multi-agent applications.pick this instead when AutoGen's actor-style components already fit your team's architecture.

What people are saying

  1. [velocity-scout] openai/openai-agents-python

Sources

  1. OpenAI Agents SDK README at commit 544b8b0
  2. Official OpenAI Agents SDK quickstart
  3. OpenAI Agents SDK model guide
  4. OpenAI Agents SDK tracing guide
  5. OpenAI Agents SDK sandbox clients
  6. OpenAI Agents SDK v0.22.1 release
  7. Windows sandbox symlink report

More llm tools reviews

gpt4free · notfair-plugin · freecad-mcp · chat-on-steroids · my-free-code · claude-cookbooks · the whole board →