mrkeyoor.com_
Wed 16 Sept 16:01 UTC
LLM Toolsevaluationupdated 16 Sept 2026

agent-router review

Envoy AI Gateway is now Agent Router, an Envoy-based control plane that gives applications one OpenAI-compatible route to hosted models, self-hosted inference, and MCP servers. It centralizes credentials, routing, quotas, failover, and usage records while Envoy handles the traffic.

+84stars / 7d
Verdict

Our Agent Router build succeeded in 273 seconds, but its 12-package test sweep ended 6 passed and 6 failed after 668 seconds, so teams with a green-suite gate should wait or reproduce the failures first. It is a sensible trial for a Kubernetes platform group that already trusts Envoy and needs one policy layer for models and MCP servers. A team with one provider will spend more attention on the control plane than it gets back.

We ran it

Lab card: what happened when we ran agent-routerScreenshot of agent-router (theagentrouter.ai)
Install✓ · 96s541 packages
Build✓ · 273s
Tests✗ · 668s6 passed · 6 failed of 12 (go test)
Repo1558 files~221,489 lines of source · 35.2 MB · 7 CI workflows · Dockerfile · tests dir

Answers from our run

Does agent-router build from source?

Dependencies installed in 96 seconds (541 packages), and the build succeeded in 273 seconds. We cloned commit c217da8 into a clean Debian container with 3 CPUs and no project-specific setup.

Do agent-router's tests pass?

Not all of them: 6 of 12 passed and 6 failed when we ran the project's own test command (go test). Some failures need services or credentials a bare container does not have.

Who should not use agent-router?

Small teams sending traffic to one model provider: the full production path adds Envoy Gateway, custom resources, and Kubernetes operations that a direct SDK call avoids.

What are the alternatives to agent-router?

LiteLLM, Portkey AI Gateway, Kong Gateway. Our Agent Router build succeeded in 273 seconds, but its 12-package test sweep ended 6 passed and 6 failed after 668 seconds, so teams with a green-suite gate should wait or reproduce the failures first.

Setup2/5Build passed, but tests split 6 passed and 6 failed
Docs4/5Local and Kubernetes paths are clear despite split naming
Community5/52,105 stars and issue activity through September 2026
Maturity4/5Stable 1.x API, with current MCP gaps and test failures

Who it’s for

Platform teams already running Envoy Gateway and Kubernetes for shared AI access.
Organizations that need provider credentials and quotas enforced outside application code.
Operators routing requests across hosted providers and self-hosted model clusters.
Teams aggregating MCP servers behind one governed endpoint.

Who it’s NOT for

Small teams sending traffic to one model provider: the full production path adds Envoy Gateway, custom resources, and Kubernetes operations that a direct SDK call avoids.
Organizations that require every clean-container test sweep to pass before evaluation: our 12-package Go test run ended with 6 failures and no specific failing assertion in the supplied log tail.
MCP operators who need cross-namespace backends today: issue 2385 says MCPRoute currently requires its backend Services to share its namespace.
Teams that cannot absorb a naming transition: the project is now Agent Router, while CRDs, image names, the aigw CLI, namespace, and Go module keep the Envoy AI Gateway names.
Strict MCP clients depending on cache metadata until issue 2665 is resolved: that report says merged list responses can contain an invalid empty cacheScope.

Setup reality

Our commit c217da8 checkout installed 541 Go packages in 96 seconds and built successfully in 273 seconds. Tests failed after 668 seconds: 6 of 12 package results passed and 6 failed. The log tail only ends in FAIL; it does not identify the failing assertions, so we cannot assign a cause.

A local trial can use the aigw binary and a provider credential, with no Docker or Kubernetes required on Linux or macOS. The production guide instead deploys the control plane on Kubernetes with Envoy Gateway, then adds provider credentials, routes, policies, and observability.

The 35.2 MB checkout contained 1,558 files and about 221,489 source lines, plus 7 CI workflows, a Dockerfile, and a tests directory. Existing deployments keep the old CRD, image, namespace, CLI, and module names after the Agent Router rebrand.

Agent Router keeps the old deployment names

The repository called Envoy AI Gateway now presents itself as Agent Router. The move is organizational and editorial, not a manifest migration: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, the aigateway.envoyproxy.io API group, aigw, and the envoy-ai-gateway-system namespace keep their names. Container images and the Go module also retain their existing paths. That protects installed clusters, but a new evaluator has to translate between the current product name and old identifiers throughout the stack.

The product gives applications one OpenAI-compatible endpoint for hosted providers, self-hosted inference, and MCP servers. Envoy carries requests while the control plane applies credentials, routing, quotas, failover, and usage attribution. The README describes a 2-tier model: a central gateway handles authentication and global policy, while a second gateway controls traffic entering a model-serving cluster. This is infrastructure for a platform team, not a convenience wrapper around one API.

The local command hides a much larger production job

A developer can set a provider credential and run aigw run, then aim an OpenAI-compatible client at port 1975. The current guide says this local mode needs neither Docker nor Kubernetes on Linux and macOS. It can also front Ollama, vLLM, and MCP servers, which makes it useful for proving a route and configuration before a cluster rollout.

Production is where the choice gets expensive. The documented path installs Agent Router as a control plane on Kubernetes with Envoy Gateway, then asks operators to manage credentials, gateway resources, routing policies, quotas, and telemetry. Version 1.1.0 adds per-request credentials, stream idle timeouts with failover, HTTP CONNECT egress, GenAI tracing, and MCP hostname selection. Those are serious platform controls, but each one creates configuration and failure modes that a direct provider SDK never introduces.

What happened when we ran it

Our sandbox installed 541 Go packages in 96 seconds and built commit c217da8 successfully in 273 seconds. The checkout itself was 35.2 MB, with 1,558 files and about 221,489 lines of source. We found 7 CI workflow files, a Dockerfile, and a tests directory. This is a substantial Go control plane, although the clean build shows its compilation path worked in the stated Debian container.

Tests were a different result. The Go test command ran for 668 seconds and finished with exit code 1: 6 package results passed and 6 failed out of 12. The supplied tail shows successful packages such as the OpenAI, OpenInference, MCP test server, and upstream test helpers, then ends with a bare FAIL. It contains no failed assertion or setup error. We can report the split, but the log does not support blaming credentials, missing services, or a particular package.

MCP aggregation still has namespace and response gaps

Agent Router can combine MCP backends and advertise merged capabilities through one route. Version 1.1.0 added hostname scoping and CEL-based backend selection, with a default deny action. This belongs at the gateway when several teams publish tools and one platform group must decide which client may see which backend. The same release also records MCP resource URIs in access-log metadata.

Two open reports matter before that design reaches a strict production client. Issue 2385 says an MCPRoute can currently reference Services only in its own Kubernetes namespace, making cross-team aggregation awkward without proxy Services or a shared namespace. Issue 2665 reports that merge functions can drop cache metadata and emit an empty cacheScope, which strict clients reject. These are specific limits, not reasons to dismiss the entire LLM routing path, but MCP buyers should reproduce both cases.

Version 1.1 improves control without making billing exact

The August 21, 2026 release supports token-counting routes across several provider formats, provider credentials supplied per request, forward-proxy egress, and retry behavior when a stream stalls before its first token. It also accepts Responses API tool-search inputs and exposes JSON log options. Upgrading from 1.0 requires no CRD migration, according to the release notes, though the Helm chart now applies a restricted controller security context by default.

Cost attribution still has a documented edge. Issue 2694 says the gateway combines Anthropic 5-minute and 1-hour prompt-cache writes under one cost type even though their rates differ. A team charging usage back to users or departments cannot recover that split from the current per-request signal. External billing exports can fill part of that gap, but they lose the request identity that makes gateway accounting useful.

September activity is high, and so is the surface area

GitHub showed 2,105 stars, 292 combined issues and pull requests, and a last push on September 15, 2026. The open queue was active again on September 16, including work on quota windows, MCP protocol handling, provider authentication, and controller readiness. The combined count is not a defect total. It does show how many providers, protocols, and Kubernetes behaviors this project is trying to hold together.

Agent Router makes sense when that breadth matches a problem you already own. Existing Envoy teams get familiar data-plane machinery, a stable 1.x API, and policy that stays outside application repositories. The failed 668-second suite is the immediate caution: reproduce it before making this gateway a release dependency. If your architecture has one model provider and no shared cluster policy, keep the direct connection and avoid creating another control plane to operate.

Alternatives

ProjectWhat it isPick it when
LiteLLM gh↗An OpenAI-compatible proxy and SDK covering many hosted model APIs.pick this instead when provider translation and spend controls matter more than Envoy and Kubernetes integration.
Portkey AI GatewayA standalone AI gateway focused on routing, fallbacks, and guardrail integrations.pick this instead when you want an AI-specific proxy without adopting Envoy Gateway custom resources.
Kong Gateway gh↗A general API gateway that also supplies AI traffic plugins and controls.pick this instead when AI routes must sit inside an existing general API management program.

What people are saying

  1. [github-trending] theagentrouter/agent-router
  2. [github-trending] envoyproxy/ai-gateway

Sources

  1. Agent Router README
  2. Agent Router getting started
  3. Agent Router v1.1.0 release
  4. Cross-namespace MCPRoute request
  5. MCP cache metadata bug report
  6. Prompt cache cost attribution request

More llm tools reviews

ouroboros · headcount · useagent · claude-skills · RAG_Techniques · ux-ui-agent-skills · the whole board →