mrkeyoor.com_
Tue 27 Oct 06:33 UTC
LLM Toolsevaluationupdated 27 Oct 2026

harnessrouter review

HarnessRouter gives applications one API for agent backends such as Codex and Claude Code. It handles the surrounding task lifecycle, including sessions, streamed progress and files, so a product can integrate several harnesses through a shared interface.

Verdict

Our HarnessRouter build succeeded in 6 seconds, but tests exited 1 and pip-audit reported 136 known vulnerabilities, leaving production readiness unproven. Evaluate its shared task API if your product needs several agent backends, after investigating the test and dependency findings. Use a direct agent when there is only one backend to integrate.

We ran it

Lab card: what happened when we ran harnessrouterScreenshot of harnessrouter (harnessrouter.ai)
Install✓ · 28s78 packages · 251 MB
Build✓ · 6s
Tests✗ · 267s1309 passed · 130 failed · 56 skipped · 3 errors of 1442 (pytest)
Known vulns136(pip-audit)
Repo914 files~149,802 lines of source · 49.8 MB · 8 CI workflows · tests dir

Answers from our run

Does harnessrouter build from source?

Dependencies installed in 28 seconds (78 packages), and the build succeeded in 6 seconds. We cloned commit b4d53a1 into a clean Debian container with 3 CPUs and no project-specific setup.

Do harnessrouter's tests pass?

No. The project's own test command finished in 267 seconds and reported 1309 passed, 130 failed, 56 skipped, 3 collection errors (pytest).

Does harnessrouter have known vulnerabilities in its dependencies?

pip-audit flagged 136 known advisories in the dependency tree at the time of our run.

Who should not use harnessrouter?

Developers who only need one terminal agent: the documented Console, Gateway and Runner add an operating layer.

What are the alternatives to harnessrouter?

Codex CLI, Goose. Our HarnessRouter build succeeded in 6 seconds, but tests exited 1 and pip-audit reported 136 known vulnerabilities, leaving production readiness unproven.

SetupN/ASource installation and build completed, but the documented container and end-to-end task setup were not reproduced.
DocsN/AThe README documents the task lifecycle and deployment boundaries; the linked support matrix and operations guide were not assessed in full.
CommunityN/ARecent releases and issue activity were inspected, but response times and long-term maintainer capacity were not assessed.
MaturityN/AFailed tests, unreconciled counts and audit findings leave operational behavior and dependency risk insufficiently assessed for a numeric rating.

Who it’s for

Product teams integrating several agent harnesses through one task API.
Operators prepared to manage provider credentials and persistent agent workspaces.
Developers who need sessions, generated files and cancellation as part of their application integration.

Who it’s NOT for

Developers who only need one terminal agent: the documented Console, Gateway and Runner add an operating layer.
Teams requiring a separate container for every session: the README specifies separate workspaces and operating-system users instead.
Organizations equating self-hosting with offline inference: the README says model requests still go to the configured provider.
Teams unable to allow the container entrypoint and Runner to manage session users as root, which the quickstart requires.

Setup reality

Our October 26, 2026 run at b4d53a1 recorded installation success in 28 seconds, 78 dependencies and 251 MB on disk, then build success in 6 seconds. Tests exited 1 after 267 seconds: pytest reports 1,309 passed, 130 failed, 56 skipped and three collection/setup errors. The stated total of 1,442 does not reconcile with those categories. The tail repeatedly reports unsupported async def functions; pip-audit reports 136 known vulnerabilities without package or severity details.

The Python project lives in gateway/. The current README instead leads users through a Docker deployment, a provider API key and persistent storage. Console credentials, provider keys and product API keys serve separate purposes. No end-to-end task execution or container deployment was measured here.

The recorded configuration was 3 CPUs, 8 GB RAM and ghcr.io/astral-sh/uv:python3.12-bookworm; actual runtime versions require logs. The README requires a root entrypoint and Runner for session users. Sessions use separate users and workspaces, not separate containers.

HarnessRouter earns its place when your product needs several agent backends

Switching an application between Codex and Claude Code involves more than changing a model name. HarnessRouter's README describes one API for persistent sessions, progress streams, files, cancellation and structured failures across agent harnesses. Those responsibilities are a reason to evaluate an intermediary when your product needs more than one agent backend.

For someone who simply wants an agent in a terminal, this is extra infrastructure. Community Edition includes a Console, Gateway and Runner, plus persistent state. Its appeal is to the application developer who would otherwise implement the same task lifecycle separately for each harness. Our sandbox did not exercise that lifecycle, so the recommendation remains a bounded integration trial.

The documented API is compatible with OpenAI Responses and selects a harness through request metadata. That describes an integration surface, not proof that every harness behaves identically. The README points to a provider support matrix, which should be checked for the particular combination your product needs.

What happened when we ran it

Our sandbox recorded installation success in 28 seconds and build success in 6 seconds at commit b4d53a1 on October 26, 2026. The Python project lives under gateway/. The record lists 78 dependencies occupying 251 MB. Tests exited 1 after 267 seconds, so successful setup did not lead to a passing suite.

The pytest summary reports 1,309 passed, 130 failed, 56 skipped and three collection/setup errors. The record also lists a total of 1,442, which does not reconcile with those categories. We therefore report the categories without presenting a passing fraction. The log's own elapsed time is 260.90 seconds; the outer recorded test step is 267 seconds. These refer to different scopes of elapsed time.

The supplied tail repeatedly says that async def functions are not natively supported. Named failures include session-file fallback and workspace-write durability cases. That is an explicit test-execution limitation visible in this run. The tail does not establish which configuration or package caused it, explain every failure or demonstrate that the corresponding application behavior is defective. We would need the complete failure logs and a supported test setup to resolve those questions.

The pip-audit result reports 136 known vulnerabilities. The supplied block does not identify packages, severity, reachability or fixes, so the count cannot establish exploitability or a remediation plan. It does make dependency review a prerequisite to a production recommendation. We cannot infer that the published container has precisely the same dependency exposure as this source environment.

The configured sandbox had 3 CPUs, 8 GB of RAM and image label ghcr.io/astral-sh/uv:python3.12-bookworm. That label alone establishes neither the actual Python version nor the operating-system version. The checkout contained 914 files, approximately 149,802 source lines and 49.8 MB. The scan found eight CI workflow files and a tests directory but no Dockerfile. The current README's container instructions are a separate deployment route that our scan and run did not validate.

Self-hosting still sends requests to your model provider

Community Edition's README requires Docker and a provider API key, with no HarnessRouter account required. The provider credential authorizes model requests. Product integration uses a separate HarnessRouter API key, issued by the local instance, while the Console has its own sign-in credentials. Keep these distinct when deciding how your backend will authenticate.

The README says provider keys and session state stay under your control, but model requests still go to the configured provider. Self-hosting the gateway therefore does not make inference offline. A team with a requirement that task content never leave its infrastructure must evaluate the provider route as well as the gateway deployment.

The documented quickstart binds the Console to loopback and warns users to change the default password before making the instance accessible. Its named data volume preserves the database, files, installed harness CLIs and workspaces. A useful deployment trial should therefore include a restart and a recovery check for the state your application needs. Neither operation was measured in our sandbox.

Session users are not separate containers

The README explicitly says sessions use separate workspaces and operating-system users rather than separate containers. It also says the entrypoint and Runner need root to manage those users, while the Console and Gateway run unprivileged. Organizations requiring a container boundary for every untrusted task should stop at that architectural distinction and evaluate a suitable deployment model before proceeding.

This is more consequential than the number of available harnesses. An agent can run shell commands and handle files, so the required separation between your users' tasks belongs in the adoption criteria. Our source build provides no independent isolation or credential-redaction validation.

The repository metadata records a push on October 27, 2026, and recent issue activity includes a report about file links opening as missing unless a task is shared. Release v0.37.3, dated October 26, describes fixes for fast-agent MCP addresses and docker-agent skill text. Together these show active changes and reported problems. They do not establish support responsiveness or eliminate the gaps in our run.

A direct agent is simpler when there is only one backend to call

Codex CLI describes itself as OpenAI's coding agent running locally on your computer. Evaluate it directly when that is the agent you need and a shared multi-harness product API adds no benefit. This is a choice of integration scope, not a comparative setup or speed result.

Goose describes a general-purpose agent running on your machine, with a desktop app, CLI and API. Its current repository is aaif-goose/goose. Consider it when you want an agent's own interface instead of operating a router across several harnesses. For HarnessRouter, the next useful proof is a representative task that continues a session, returns a generated file and cancels correctly through your backend, after the test and dependency findings have been investigated.

Alternatives

ProjectWhat it isPick it when
Codex CLI gh↗OpenAI's local coding agent.Choose Codex directly when it is the one agent you need and a shared multi-harness product API adds no benefit.
Goose gh↗A local general-purpose agent with desktop, CLI and API interfaces.Evaluate Goose when you want one agent interface rather than a router across several harnesses.

What people are saying

  1. [github-trending] HarnessRouter/harnessrouter

Sources

  1. HarnessRouter README
  2. Codex CLI README
  3. Goose README
  4. HarnessRouter v0.37.3
  5. HarnessRouter repository facts
  6. HarnessRouter issue and PR activity

More llm tools reviews

pi-subagents · kimi-code · RAG-Anything · uzu · dsh-im · smg · the whole board →