A traffic controller for mixed model stacks
Switchyard sits between an AI client and one or more language-model backends. The client keeps speaking OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages. Switchyard selects a configured target, converts the request into that target's format, and translates the answer back. That sounds like an ordinary proxy until the client is Claude Code or Codex and the backend is an open model served by vLLM, NVIDIA NIM, Ollama, or another OpenAI-compatible endpoint. Then protocol translation is the difference between a quick experiment and rewriting the client.
Routing is the other half. A random strategy can split traffic for an A/B test. An LLM classifier can send ordinary prompts to a cheaper model and harder prompts to a stronger one. A stage router looks at tool results, errors, and conversation progress. Escalation first runs the weaker model, then asks a judge whether the answer deserves another attempt. These are useful primitives for teams trying to control model cost without pinning every application to one provider.
The proxy also exposes health checks, model discovery, Prometheus metrics, OpenTelemetry spans, aggregate statistics, optional routing logs, retries, and context-window recovery. Switchyard is therefore more than a format converter, but it is not a complete hosted control plane. You still choose, pay for, secure, and operate the upstream models.
Three entry points, with different commitments
The quickest path is the Python-distributed launcher. Install the nemo-switchyard command with uv, provide an OpenRouter key, and launch Claude Code, Codex, or OpenClaw against the packaged deployment. That is a sensible way to learn what routing changes in practice. It does require Python 3.12, uv, the chosen coding agent, and its normal prerequisites. The launcher starts a packaged native server binding; it does not install the standalone server binary.
Running Switchyard as infrastructure is a bigger step. Cargo builds switchyard-server, and an explicit TOML file defines upstream clients, credentials, targets, route IDs, and algorithms. The dry-run option catches configuration and environment mistakes before binding a port, which is a thoughtful operational detail. A real rollout still needs TLS decisions, secret management, metrics collection, provider limits, and tests for every protocol conversion your agents use. Pointing at local vLLM or Ollama also means operating those model servers separately.
The third path is the most distinctive. switchyard-libsy provides routing algorithms without owning an HTTP server or provider SDK. Algorithms yield model-call steps to the host, so an existing Rust gateway can keep its own transport, retry logic, and credentials. This boundary is attractive for infrastructure developers, but it is not plug-and-play. The host must execute those steps correctly and deal with streaming, errors, state, and concurrent calls.
Translation is both the feature and the risk
Normalizing three related APIs is harder than matching endpoint names. Tool definitions, tool calls, tool results, reasoning items, usage details, cached-token counts, stop reasons, streaming events, and error envelopes all carry slightly different meanings. Release 0.2.0 says Switchyard preserves many of these details and validates known request fields instead of silently discarding them. That is the right design goal for coding agents, whose multi-step tool conversations stress far more of an API than a basic chat box.
Current issues show why the project still warns users away from production. Issue #380 reports that image and document blocks inside tool results become a text string when translated to OpenAI Chat. Issue #369 reports that an OpenAI content-filter stop becomes an Anthropic end_turn, making a refusal look like an ordinary completion. A fix for a separate nested JSON-schema problem landed within about an hour on August 12, which demonstrates responsiveness, but fast fixes do not erase the need for compatibility tests.
The release notes list more known gaps. Work upstream may continue after a client disconnects and still cost money. Some routing decisions lack target attribution in metrics, a recovery counter can remain zero after a successful retry, and native session statistics omit a supplied session ID. None makes the project useless for a lab or benchmark. Together they make it difficult to treat observability as an audit record.
A young project moving very quickly
Version 0.2.0 arrived on August 10, 2026, after 193 commits since 0.1.0. It rebuilt the project around a native Rust server and the new library architecture. It also deprecated the Python server, YAML route bundles, and profile APIs, while removing several older routers and CLI workflows. That is healthy iteration for pre-alpha software and expensive churn for anyone who integrated the first design.
The repository was pushed on August 12, with issues and pull requests updated the same day. Its 81-item open count includes both, so it is not a count of confirmed defects. The activity is unusually high for a repository created in May, and reports are receiving code changes quickly. Community size and long-term maintenance habits, however, cannot be inferred from a few intense months.
Documentation is already strong. The README distinguishes launcher, server, and library paths; the getting-started guide explains a full TOML route; individual algorithm and crate guides cover deeper behavior. The blunt maturity warning is valuable too. Switchyard is worth trying for model-routing experiments and custom Rust infrastructure. For production agent traffic, wait for a stable contract and prove every translation path with your own recorded conversations before reconsidering.