Routing decisions are the reason to evaluate Switchyard
NVIDIA NeMo Switchyard selects which model handles an agent request. Its current README puts the Rust library libsy at the center: your gateway or harness owns model clients, credentials, and service operation. Switchyard supplies the routing decision.
That division matters if you already operate a gateway. You can evaluate routing without handing the entire service to Switchyard's standalone server. The maintainers label libsy beta for trial integrations, the LLM client and runner alpha, and the server a demo. The current README says the standalone server is not a supported production deployment path.
The project is pre-1.0. Pin the version and configuration you evaluate. The v0.2.0 release described a redesign around separate Rust components, including translation for OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. Those release notes describe that version; they do not establish the status of every limitation in current code.
What our recorded run establishes
Our recorded run took place on 13 October 2026 at commit f26dafe42b1eb0bf9b3f9e36a96942b485841582. It used 3 CPUs and 8 GB of memory; the install tail identifies Python 3.12.12.
The install step exited 0 after 395 seconds. The record lists 36 dependencies and 38 MB. A harness step labeled build exited 0 after 10 seconds, but its output tail is empty. We cannot identify the build command or its artifact from this record. This is not proof of a successful Rust compilation, installed CLI, or running proxy. A later test explicitly failed because cargo was unavailable.
The pytest command exited 1 after 32 seconds of recorded step time. Its own summary reports 26.52 seconds, with 153 passed, 18 failed, 1 skipped, 1 warning, 2 errors, and 2 subtests passed. The parser records total 173. These summaries have not been reconciled against a complete log to establish exhaustive unique test coverage, so a pass percentage would overstate the evidence.
The supplied failure lines identify missing async-test support across the listed async failures and a missing cargo executable in the CLI-reference test. The tail also names errors in tests/e2e/test_closed_book_proxy_integration.py and tests/test_prepare_harbor_dataset.py, without showing their causes. These observations establish an unsuccessful command in this environment. They do not establish that Switchyard's routing is defective.
The checkout record lists 680 files, approximately 158,625 source lines, 10.9 MB, 8 CI workflows, a Dockerfile, and a tests directory. These describe the checkout, not operational reliability. Pip-audit recorded 0 advisories, with unknown coverage. No security conclusion follows from that total.
We did not record an installed-binary check, a running-server check, live provider requests, agent-task completion, or a measured routing workload.
Choose an integration before choosing an algorithm
The current README presents Auto, Task, Execution, and Composite as the starting choices. Auto is a fixed preset in v0.3.0: Execution (Stage), efficient-first, with a confidence threshold of 0.5 and no classifier call. It does not compare routing strategies at runtime. Task asks a model to judge whether the efficient model can handle a task; Execution uses recent tool activity and outcome signals; Composite combines task judgment with execution signals.
Integration support differs. The Switchyard README calls its LiteLLM example experimental and checkout-only, pins LiteLLM 1.102.0, and limits the example to Stage and Random routing. It cannot service the intermediate model calls required by classifier or escalation algorithms. LiteLLM is therefore both a gateway option and a possible host for a limited Switchyard integration.
NeMo Relay requirements also depend on the source version. The README pinned to our measured commit specifies Relay >=0.8.0, <1.0.0; the supplied current README specifies >=0.10.0, <1.0.0. Both warn that Relay 0.8.x and 0.9.0 can lose upstream error status and details when the plugin is enabled. Use the requirements for the exact checkout you integrate rather than combining instructions from different revisions.
The existing command cargo install --locked switchyard-server is an installation example, not a command verified by our run. It also leaves the version unpinned. Select an explicit version before a reproducible evaluation, and keep the standalone server within its documented demo and local-evaluation scope.
Historical reports need version checks
Issue 480 described lost upstream response headers, but it was closed as completed through PR 571 on 14 September 2026. It should not be presented as proof that current Switchyard still discards all upstream headers. Closure also does not establish that every header is forwarded by every release; verify the behavior your integration requires.
PR 560 described Codex tool-signal parsing problems involving exec_command, encoded arguments, and the cmd field. The author closed it on 27 August 2026 after stating that the work would be split into separate PRs. Its recorded-session observations belong to that contributor, not our lab run. This closed proposal alone establishes neither a current defect nor a shipped fix.
The v0.2.0 release notes separately list continued buffered work after client disconnects, incomplete target attribution in some statistics, a retry-recovery counter problem, and missing session IDs in native session statistics. Treat these as release-specific warnings to check against the version you deploy, rather than an inventory of verified current defects.
Evaluate completed tasks before claiming savings
The README warns that a cheaper model call does not guarantee a cheaper successfully completed task. Our run provides no savings or routing-performance benchmark.
For a useful next evaluation, pin the agent, model pool, Switchyard version, and configuration. Compare routing with a single-model baseline on the same tasks. Record task success, total provider cost, elapsed time, and which model actually served each request. Include tool calls, streaming, provider errors, and client cancellation where those behaviors matter to your application. These are proposed checks, not completed tests.
For a centralized gateway, LiteLLM documents a Python SDK and proxy service. Portkey AI Gateway documents retries, fallbacks, load balancing, and conditional routing. Those are relevant choices when the surrounding gateway is the main need; neither was tested in our Switchyard run.
How to read the scores
Scores use a five-point editorial rubric: 1 means insufficient support or a substantial documented constraint; 2 means limited evidence or substantial integration work; 3 means a documented path with material qualifications; 4 requires strong evidence across the relevant use; 5 requires extensive independent validation. Unknowns do not earn a high score. These ratings assess the evidence available for this review, not adoption, security, or production reliability.
Switchyard is worth a bounded integration trial when model selection is the problem you need to solve. First establish that the pinned integration runs correctly in your environment, then measure the complete agent workflow.