mrkeyoor.com_
Tue 01 Sept 17:43 UTC
LLM Toolsevaluationupdated 27 Aug 2026

aisuite review

aisuite is a Python library that gives several hosted and local language-model providers one OpenAI-shaped chat interface. It also has an agent runner, tool policies, persistent state, streaming, and MCP connections for developers who want one application to switch among providers.

+40 / 4dstars / 7d
Verdict

Our aisuite run installed 136 packages in 78 seconds, but 43 tests failed with repeated messages that async test functions were not natively supported. Use it for a Python application that genuinely needs provider switching or MCP tools and can maintain provider-level contract tests. A single-provider service is usually better served by that provider's official SDK, while automatic tool loops need extra scrutiny before production use.

We ran it

Lab card: what happened when we ran aisuiteScreenshot of aisuite (github.com/andrewyng/aisuite)
Install✓ · 78s136 packages · 521 MB
Build✓ · 7s
Tests✗ · 48s435 passed · 43 failed · 47 skipped of 478 (pytest)
Known vulns1(pip-audit)
Repo747 files~115,616 lines of source · 7 MB · 2 CI workflows · tests dir

Answers from our run

Does aisuite build from source?

Dependencies installed in 78 seconds (136 packages), and the build succeeded in 7 seconds. We cloned commit 531b5b8 into a clean Debian container with 3 CPUs and no project-specific setup.

Do aisuite's tests pass?

Not all of them: 435 of 478 passed and 43 failed when we ran the project's own test command (pytest). Some failures need services or credentials a bare container does not have.

Does aisuite have known vulnerabilities in its dependencies?

pip-audit flagged 1 known advisory in the dependency tree at the time of our run.

Who should not use aisuite?

Teams that require automatic tool loops to behave identically across providers: open issue 369 reports that max_turns fails in different ways with Groq and Ollama.

What are the alternatives to aisuite?

LiteLLM, Pydantic AI, OpenAI Python. Our aisuite run installed 136 packages in 78 seconds, but 43 tests failed with repeated messages that async test functions were not natively supported.

Setup3/5Install and build passed; 43 tests failed in our sandbox
Docs4/5Clear chat, agent, streaming, policy, state, and MCP examples
Community4/516,178 stars and open work updated in August 2026
Maturity3/5Wide provider surface, with unresolved tool-loop and test failures

Discussed on

  1. hnAndrew Ng ships Open CoWorker desktop AI agent5 points
  2. hnAisuite – Simple, unified interface to multiple Generative AI providers4 points

Who it’s for

Python teams comparing or routing calls across OpenAI, Anthropic, Google, Ollama, and other providers.
Agent developers who want callable Python tools, approval policies, run persistence, and tracing in one library.
MCP client builders who want server tools exposed through the same tool-calling loop.
Applications willing to test every provider and model combination they ship.

Who it’s NOT for

Teams that require automatic tool loops to behave identically across providers: open issue 369 reports that max_turns fails in different ways with Groq and Ollama.
Developers who need streaming plus automatic tool execution in one call: the README says streamed tool calls are manual and cannot be combined with max_turns.
Projects that refuse provider-specific packages or credentials: the base install excludes provider SDKs, and hosted services require their own API keys.
Buyers seeking a finished desktop coworker in this repository: the README says OpenWorker development moved to a separate project, leaving only a historical snapshot here.

Setup reality

Our sandbox install succeeded in 78 seconds, adding 136 packages and using 521 MB. The build passed in 7 seconds. Tests then failed with exit code 1 after 48 seconds: pytest reported 435 passed, 43 failed, 47 skipped, and 27 warnings. Pip-audit found 1 known vulnerability.

Hosted providers need their own API keys and optional SDK extras. Ollama is the documented local route. MCP support is another extra, while persisted agent runs can use memory, files, or Postgres.

The checkout had 747 files and about 115,616 source lines at commit 531b5b8. It had 2 CI workflows and a tests directory, but no Dockerfile. The failing log repeatedly said async test functions were not natively supported; it did not identify why that support was absent.

One Python interface now covers chat, agents, and MCP

aisuite began with a simple proposition: use an OpenAI-shaped chat API while selecting the provider through a string such as openai:model or anthropic:model. The current README goes further. Plain Python functions can become tools, max_turns can run the tool loop, and a separate Agent and Runner layer adds policies, saved state, artifacts, and tracing. MCP servers can enter the same tool list through an optional package.

That breadth changes the buying decision. This is no longer only a convenience adapter for a weekend model comparison. Our checkout contained 747 files and roughly 115,616 lines of source. An application can depend on aisuite for request translation, streamed chunks, tool schemas, approvals, and conversation state. Each layer saves application code, but it also makes provider differences harder to see when a request goes wrong.

Provider switching works best with a contract test for each model

The chat interface standardizes messages and familiar controls such as temperature, maximum tokens, and tools. Provider extras keep the base package smaller, and adding an adapter follows a documented module and class naming convention. This is useful when a team wants to compare models without rewriting every call site or needs an Ollama option beside hosted APIs.

The common shape does not guarantee common behavior. The README limits documented streaming support to OpenAI, Anthropic, Ollama, and OpenAI-compatible endpoints. Streamed tool calls return fragments that the caller assembles and executes manually, and streaming cannot be combined with max_turns. Those are reasonable boundaries, but they mean a provider swap needs tests for streaming, tool arguments, stop reasons, errors, and any model-specific parameters your product uses.

What happened when we ran it

Our sandbox installed aisuite in 78 seconds, pulling 136 packages and occupying 521 MB on disk. The build completed in 7 seconds. Pip-audit reported 1 known vulnerability in the installed environment. These figures come from commit 531b5b8 in a fresh unprivileged Debian container with 3 CPUs, 8 GB of RAM, Python 3.12, and no secrets.

Tests failed with exit code 1 after 48 seconds. Pytest's final summary was 435 passed, 43 failed, 47 skipped, and 27 warnings in 38.07 seconds. The log tail showed failures across Gemini, Google, OpenAI, streaming, and async tool utilities. Each displayed failure ended with the same plain message: async functions are not natively supported. The log does not tell us why that support was missing, so we cannot fairly label the library's async implementation broken from this run alone.

Repository signals are mixed in a useful way. There were 2 CI workflow files and a dedicated tests directory, but no Dockerfile. The install and build were clean, and hundreds of tests passed. Still, a developer evaluating async clients or streams should reproduce the suite in the intended environment before trusting those paths. Forty-three failures are a result to investigate, even when they share what may be a harness-level message.

Automatic tool loops still expose provider differences

Passing max_turns lets aisuite call a model, execute requested functions, return their results, and continue. The response retains intermediate messages, which is useful for resuming or auditing a run. The Agents API builds on that loop with file, git, and shell toolkits. It also supplies allow and deny lists plus approval policies, controls that matter when model output can trigger code or filesystem actions.

Open issue 369 gives a concrete warning. Its reporter found automatic tool execution failing with Groq because of an unsupported reasoning_content field and with Ollama because tool calls came back empty. The same report says a manual Ollama loop worked. That does not prove every model or current branch behaves the same way, but it does show why the 2-line provider switch needs integration coverage when tools can change files, call services, or run shell commands.

MCP support makes the security boundary wider. A configuration can launch an MCP server with a command and arguments, then let the selected model request those tools. The explicit client supports reusable connections, filtering, and tool prefixes. Those facilities help, but the application owner still decides which executable runs, what directory or service it can reach, and which calls require approval. Installing the MCP extra is the easy part.

Current work is active, while the release record is untidy

GitHub showed 16,178 stars and 147 combined issues and pull requests when we fetched the repository. The last push was August 13, 2026, and an open provider pull request was updated on August 27. That combination indicates current contribution activity; the 147 figure is not a bug count. The MIT license is straightforward for commercial and private use.

The latest-release endpoint returned tag v0.1.3, published July 20, 2026, with the name OpenWorker 0.1.3. Its notes span aisuite provider work and a long set of OpenWorker changes, even though the README says OpenWorker now lives in its own repository. Releases remain usable evidence of activity, but the naming and mixed scope make them a poor guide to which Python-library changes belong to one install. Pin a tested commit or package version rather than reading the release title as a compatibility promise.

Use the abstraction where switching has a measurable payoff

A team comparing several hosted models can get real value from one client, especially when it also needs local Ollama, MCP servers, and approval-aware Python tools. The documentation gives enough code to understand the boundaries, and the provider adapter convention is approachable. The cost is a larger behavior matrix than the unified method signature suggests.

Our 78-second install and 521 MB environment make a trial cheap enough, but the failed async-heavy test run should be resolved in the team's own CI before adoption. For one provider, its official SDK keeps errors and features closer to the source. For several providers, aisuite earns a trial if you are prepared to test the exact models, streams, and tool loops users will depend on.

Alternatives

ProjectWhat it isPick it when
LiteLLM gh↗A multi-provider SDK and proxy with routing, budgets, and an OpenAI-compatible gateway.pick this instead when a shared network proxy and centralized provider controls matter more than an in-process Python agent API.
Pydantic AIA typed Python agent framework built around model providers and validated outputs.pick this instead when typed dependencies, output validation, and agent structure are more important than a small chat-completions adapter.
OpenAI PythonThe official Python client for OpenAI's APIs.pick this instead when your application uses only OpenAI and provider portability would add another abstraction to debug.

What people are saying

  1. [github-trending] andrewyng/aisuite

Sources

  1. aisuite README
  2. aisuite repository facts
  3. aisuite latest release
  4. Issue 369: Groq and Ollama tool calling

More llm tools reviews

rig · open-knowledge · graphiti · cve-mcp-server · minimind · SillyTavern · the whole board →