mrkeyoor.com_
Thu 10 Sept 18:02 UTC
LLM Toolsevaluationupdated 10 Sept 2026

agent-router review

Agent Router is the new name for Envoy AI Gateway, an open-source control plane that puts one OpenAI-compatible address in front of hosted models, self-hosted inference, and MCP servers. It centralizes provider credentials, routing, quotas, fallback, and usage records while Envoy handles the network traffic.

trackingstars / 7d
Verdict

Our Agent Router build completed in 317 seconds, but its 653-second test run failed 16 of 28 reported tests, so source adopters need Kubernetes upgrade coverage of their own. Use it when AI routing has become a platform responsibility and Envoy Gateway is already acceptable infrastructure. A small app forwarding to one provider should choose a narrower proxy, while strict MCP users should reproduce the open metadata bug before rollout.

We ran it

Lab card: what happened when we ran agent-routerScreenshot of agent-router (theagentrouter.ai)
Install✓ · 67s532 packages
Build✓ · 317s
Tests✗ · 653s12 passed · 16 failed of 28 (go test)
Repo1629 files~227,007 lines of source · 31.3 MB · 7 CI workflows · Dockerfile · tests dir

Answers from our run

Does agent-router build from source?

Dependencies installed in 67 seconds (532 packages), and the build succeeded in 317 seconds. We cloned commit 42a20b9 into a clean Debian container with 3 CPUs and no project-specific setup.

Do agent-router's tests pass?

Not all of them: 12 of 28 passed and 16 failed when we ran the project's own test command (go test). Some failures need services or credentials a bare container does not have.

Who should not use agent-router?

Teams on Kubernetes older than 1.32 or Envoy Gateway older than 1.8.1: the current compatibility page does not support those combinations.

What are the alternatives to agent-router?

LiteLLM, Portkey AI Gateway, Helicone. Our Agent Router build completed in 317 seconds, but its 653-second test run failed 16 of 28 reported tests, so source adopters need Kubernetes upgrade coverage of their own.

Setup2/5Build took 317 seconds; 16 tests failed and Kubernetes adds work
Docs5/5Local, Kubernetes, provider, MCP, quota, and upgrade paths are covered
Community5/52,032 stars with code and issue activity on September 10, 2026
Maturity3/5v1.1.0 is stable; upgrade tests failed and current bugs remain

Who it’s for

Platform teams already running Kubernetes and Envoy Gateway for shared AI traffic.
Organizations that need one policy layer across hosted providers and self-hosted models.
Operators who want token quotas, provider fallback, and OpenTelemetry data enforced at the gateway.
MCP platform owners who need to combine servers, filter tools, and apply OAuth or API-key policies.
Developers who want a local OpenAI-compatible router through the standalone aigw binary.

Who it’s NOT for

Teams on Kubernetes older than 1.32 or Envoy Gateway older than 1.8.1: the current compatibility page does not support those combinations.
Small applications that only need to proxy one provider key: production setup adds CRDs, Helm releases, Envoy Gateway, and optional Redis.
Strict MCP clients relying on current cache metadata: issue 2665 reports an invalid empty cacheScope that can break tool discovery.
Filter chains that mutate request bodies before Agent Router rewrites them: issue 2671 reports that an upstream replacement can discard earlier mutations.
Operators requiring Kubernetes Service references directly from AIServiceBackend: issue 902 says that support is temporarily disabled.
Buyers who require a clean source test run in a plain container: our measured commit failed 16 of 28 reported tests.

Setup reality

Our sandbox installed 532 Go packages in 67 seconds and built commit 42a20b9 in 317 seconds. Tests ran for 653 seconds and failed: go test reported 12 passed and 16 failed out of 28. The log tail named TestUpgrade and TestCRDVersionUpgrade, then ended in the tests/e2e-upgrade package without exposing the cause.

The standalone aigw run path needs a provider API key or an OpenAI-compatible local backend and listens on port 1975. Kubernetes deployment needs kubectl, Helm, a cluster, Envoy Gateway, matching CRDs and charts, provider credentials, and Redis for quota or token-rate policies.

The CLI supports Linux and macOS. Current Kubernetes docs require version 1.32 or newer and Envoy Gateway 1.8.1 or newer. The project was renamed, but the aigw command, API group, namespace, images, charts, and Go module still use the Envoy AI Gateway names.

One API controls model traffic and MCP tools

The README lists 16 provider entries behind one OpenAI-compatible address. Agent Router lets platform owners choose credentials, routing rules, and limits while Envoy carries requests. The Go control plane turns resources such as AIGatewayRoute, AIServiceBackend, and BackendSecurityPolicy into traffic configuration. Those provider entries cover hosted APIs and OpenAI-compatible inference. Model virtualization lets a public model name map to different backends.

The same gateway can aggregate MCP servers into one endpoint. Tool names receive backend prefixes, policies can filter the visible set, and OAuth or injected API keys protect each route. The current documentation claims complete coverage of the June 2025 MCP specification. Open issue 1575 tracks additions from the November 2025 specification, so buyers should match the gateway's supported protocol to their clients instead of reading MCP support as timeless compatibility.

Release 1.1 adds token counting and first-byte failover

Version 1.1.0 was published on August 21, 2026 as the first minor release on the stable 1.x API. It added token-counting routes across several provider shapes, per-request upstream credentials, an HTTP CONNECT egress option, MCP hostname routing, and optional OpenTelemetry GenAI tracing. A stream idle timeout can move a request to another backend if it fires before the first token; after streaming begins, the documented result is a 504.

Quota policy goes beyond request counting. It can charge model-specific buckets by total, input, output, cached, or reasoning tokens and return 429 when applicable buckets are exhausted. Redis stores the counters, and shadow mode can record outcomes without rejecting traffic. One trap is explicit in the docs: the quota model name must match the route's modelNameOverride, or the limit is silently skipped.

Local mode uses port 1975; production uses Helm and CRDs

The standalone aigw run process listens on port 1975, and official binaries plus a Docker image provide the smallest trial. On Linux or macOS, setting an OpenAI key starts the local proxy. OpenAI-compatible services such as Ollama can be selected with a base URL. A custom YAML file uses the same resource vocabulary as the Kubernetes deployment, which makes local configuration useful before a cluster rollout.

Production installation is several layers deeper. The current guide requires Kubernetes 1.32 or newer, Envoy Gateway 1.8.1 or newer, kubectl, Helm, an Envoy Gateway deployment with project-specific values, a CRD chart, and the controller chart. Provider access adds secrets. Token quotas and usage-based limits add Redis and rate-limit configuration. OpenTelemetry can include full model request and response data, so prompt capture needs an explicit privacy decision.

The September rename did not change deployment identifiers. The executable remains aigw, the API group remains aigateway.envoyproxy.io, and the namespace, images, charts, and Go module still use Envoy AI Gateway names. Existing manifests keep working, which is sensible operationally. Teams writing inventories or policy rules around the Agent Router name must account for the older identifiers.

What happened when we ran it

On September 10, 2026, we cloned commit 42a20b9 into a fresh, unprivileged Debian container with 3 CPUs and 8 GB of RAM. Our sandbox installed 532 Go packages in 67 seconds, then built successfully in 317 seconds. The checkout contained 1,629 files, about 227,007 source lines, and occupied 31.3 MB before dependencies. It also had 7 CI workflow files, a Dockerfile, and a tests directory.

Tests ran for 653 seconds and ended with exit code 1. The supplied summary says go test reported 12 passed and 16 failed out of 28. The tail shows TestUpgrade and TestCRDVersionUpgrade failing inside github.com/envoyproxy/ai-gateway/tests/e2e-upgrade; several internal helper packages passed just before the final failure. The log excerpt gives no assertion or environmental diagnosis, so we cannot say whether cluster access, fixtures, or code caused the failures.

A 317-second successful build confirms compilation in the stated 3-CPU, 8 GB Debian container. It does not cancel a 653-second failed suite, especially when upgrade behavior is involved. Anyone modifying controllers, CRDs, or Helm output should rerun the relevant tests in the environment documented by the project and keep the full failure output.

Current MCP and filter bugs can change request meaning

Issue 2665 reports that MCP list responses can emit invalid cacheScope: "" metadata. The reporter says strict clients reject that value during tool discovery even though authentication and initialization succeed. Issue 1575 also remains open for November 2025 MCP additions. Organizations standardizing on MCP should test initialization, list operations, notifications, and tool calls with their exact client versions.

Issue 2671 reports a separate body-rewrite problem. When an upstream external-processing filter replaces the request body, it can rebuild from an earlier router snapshot and discard changes made by another filter in between. This matters for stacks that sanitize, enrich, or authorize JSON through multiple Envoy filters. Both reports are open issue evidence rather than findings from our sandbox, but each describes a failure that can alter what reaches a model or tool.

A September 10 push confirms activity, not a clean rollout

GitHub showed 2,032 stars and 282 open issues and pull requests, with a source push on September 10, 2026. New bug reports and merged changes appeared the same day. That is active maintenance by any practical reading, although the combined count is not 282 confirmed defects. Release 1.1.0 is also recent, and its notes include a specific security-context change for custom controller images.

Agent Router earns consideration when one team must govern many providers, tenants, or MCP servers through existing Envoy infrastructure. Its policy range is wider than a lightweight proxy, and the docs explain the moving parts with unusual specificity. The cost is equally plain: version-coupled Kubernetes components, old and new names in one deployment, a long test cycle, and 16 failures in our measured run. Trial the standalone CLI first, then promote only after the upgrade and protocol cases pass in your cluster.

Alternatives

ProjectWhat it isPick it when
LiteLLM gh↗A multi-provider model gateway and SDK with an OpenAI-compatible proxy.pick this instead when a simpler provider proxy matters more than Envoy and Kubernetes policy integration.
Portkey AI GatewayAn open-source AI gateway centered on provider routing and guardrail integrations.pick this instead when model routing and guardrails are the priority and Gateway API CRDs are unwanted.
HeliconeAn open-source LLM observability platform with a gateway entry point.pick this instead when request monitoring and evaluation matter more than cluster traffic policy.

What people are saying

  1. [github-trending] theagentrouter/agent-router

Sources

  1. Agent Router README
  2. Agent Router v1.1.0 release
  3. Agent Router compatibility matrix
  4. Agent Router standalone CLI guide
  5. Agent Router MCP gateway guide
  6. Issue 2665: invalid MCP cache metadata
  7. Issue 2671: earlier body mutations can be discarded
  8. Issue 902: Kubernetes Service backend references

More llm tools reviews

llm-for-zotero · webcodex · codegraph · semantic-kernel · cc-connect · no-ai-slop · the whole board →