mrkeyoor.com_
Wed 16 Sept 05:27 UTC
LLM Toolsevaluationupdated 25 Aug 2026

crewAI review

CrewAI is a Python framework for arranging language-model agents into teams and placing their work inside event-driven flows. It tackles jobs where one prompt is too limited, especially when separate roles, tools, state, branching, or human review need to share one workflow.

+323stars / 7d
Verdict

Our CrewAI checkout installed 34 packages in 29 seconds, but its build and test steps both failed, so the quick application path is more convincing than the contributor path we measured. Use CrewAI when role-based agents genuinely make a job easier to express, then wrap risky or repeatable work in Flows. Skip it for a one-model pipeline or when your team cannot own provider costs, telemetry policy, and tool permissions.

We ran it

Lab card: what happened when we ran crewAIScreenshot of crewAI (crewai.com)
Install✓ · 29s34 packages · 37 MB
Build✗ · 13s
Tests✗ · 8sran, no count parsed
Known vulns0(pip-audit)
Repo27141 files~317,240 lines of source · 359.9 MB · 14 CI workflows

Answers from our run

Does crewAI build from source?

Dependencies installed in 29 seconds (34 packages), and the build failed. We cloned commit 9652af6 into a clean Debian container with 3 CPUs and no project-specific setup.

Do crewAI's tests pass?

The test command failed in our container, and its output did not report a pass or fail count.

Does crewAI have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use crewAI?

Teams that need every step to be deterministic: the README separates autonomous Crews from Flows because agent decisions and model output need explicit workflow boundaries.

What are the alternatives to crewAI?

LangGraph, Microsoft Agent Framework, AutoGen. Our CrewAI checkout installed 34 packages in 29 seconds, but its build and test steps both failed, so the quick application path is more convincing than the contributor path we measured.

Setup3/5Fast install, but our build and test commands both failed
Docs4/5Clear starts and concepts, with contributor setup missing a dependency
Community5/5Fresh releases and active issue and pull-request work
Maturity4/5Broad workflow API, though source verification failed on our box

Who it’s for

Python teams building multi-step agent workflows with distinct roles and tools.
Developers who want autonomous Crews inside more predictable, stateful Flows.
Organizations willing to operate model credentials, tracing, and agent safety policy.
MCP users who want agents to call external tool servers through the framework.

Who it’s NOT for

Teams that need every step to be deterministic: the README separates autonomous Crews from Flows because agent decisions and model output need explicit workflow boundaries.
Developers who want a useful run with no model account or local model service: the quick start requires a provider API key, and its search example also needs a Serper key.
Organizations that prohibit usage telemetry unless it is disabled centrally: CrewAI collects anonymous runtime details by default, with OTEL_SDK_DISABLED=true as the documented opt-out.
Contributors expecting the documented source checkout to test immediately in a minimal container: our pytest run stopped because python-dotenv was missing.
Buyers who assume the MIT repository includes the enterprise control plane: deployment management, governance, and enterprise support are sold through CrewAI AMP.

Setup reality

Our sandbox installed 34 packages in 29 seconds, using 37 MB, at commit 9652af6. The build then failed in 13 seconds, and pytest stopped in 8 seconds with exit 4 because conftest.py could not import dotenv. Pip-audit found 0 known vulnerabilities.

A normal project needs Python 3.10 through 3.13, uv, and a model provider API key. The README's search example also needs a Serper key. Local models are supported through options such as Ollama, but that moves the model service onto your own machine.

The checkout was 359.9 MB across 27,141 files. CrewAI sends anonymous telemetry unless OTEL_SDK_DISABLED=true; enabling share_crew sends more detailed task and crew data. Production monitoring, governance, and managed deployment belong to the separate AMP offering.

Two abstractions split autonomy from control

CrewAI gives Python developers 2 related ways to organize model work. Crews assign roles, goals, tools, and tasks to agents that collaborate with some freedom. Flows are event-driven programs with state, branches, routing, and ordinary Python around model calls. The split is sensible. A research task may benefit from agents dividing work, while payment approval or file deletion belongs in a path whose transitions the application controls. CrewAI lets a Flow call a Crew at the point where that uncertainty is acceptable.

The current scaffold puts agent definitions in JSONC files and crew settings in crew.jsonc; a --classic option retains the older Python and YAML layout. Missing input values are requested when the crew runs. This gives teams a reasonably inspectable layer for prompts and task relationships without forcing all application logic into configuration. It also creates another format to review, version, and migrate, so a small two-call script may be clearer as plain Python.

A useful run still needs at least 1 model connection

The CrewAI README requires Python 3.10 or newer and below 3.14, with uv handling the CLI and project environment. After scaffolding, the user supplies a model provider API key. The bundled research example adds Serper for web search. CrewAI defaults to OpenAI in its example configuration, while its model guide also covers other providers and local services such as Ollama. The framework is open source; the useful inference and search behind many examples may still have a bill or another server to maintain.

CrewAI's feature list covers memory, knowledge, checkpointing, asynchronous execution, structured output, human input, and MCP or A2A connections. That is enough surface for a long-running internal process, but each capability adds failure modes outside the agent prompt. Tool calls need authorization, checkpoints need storage rules, and human review needs a timeout path. The README makes Flows the home for those decisions. Teams should resist turning every ordinary function into an agent merely because the framework can coordinate one.

What happened when we ran it

Our run cloned commit 9652af6 into an unprivileged Debian container with 3 CPUs, 8 GB of RAM, and no secrets. The checkout occupied 359.9 MB and contained 27,141 files with about 317,240 lines of source. Installation succeeded in 29 seconds, adding 34 packages and 37 MB. Pip-audit reported 0 known vulnerabilities in that installed environment.

The build failed with exit 1 after 13 seconds. The supplied log tail contains no diagnostic beyond that outcome, so we cannot responsibly assign a cause. Tests then failed with exit 4 after 8 seconds. Pytest stopped while loading the repository's root conftest.py: Python could not import dotenv, reporting ModuleNotFoundError: No module named 'dotenv'. That is a setup failure before test collection, not a failed CrewAI assertion and not evidence about how many tests pass.

Repository signals show 14 CI workflow files, no Dockerfile, and no top-level tests directory. The README's contributor command points pytest at lib/crewai/tests/, so the absence of a root tests folder is consistent with the repository layout. Still, uv sync --all-groups --all-extras did not leave our fresh environment able to import the root test configuration. Contributors should expect to reconcile the documented environment before they get a meaningful local result.

Telemetry needs an explicit 1-line policy

CrewAI says its default anonymous telemetry includes the framework and Python versions, general operating system details, agent and task counts, process type, model name, roles, and available tool names. Setting OTEL_SDK_DISABLED=true disables that collection. A separate share_crew=True option sends detailed task and execution data, including goals, context, and output. Those are materially different modes. Put the chosen setting into deployment configuration instead of leaving it to each developer's shell.

The distinction between the MIT framework and the commercial AMP suite also matters. The repository supplies the programming model. CrewAI markets managed deployment, observability, governance, security controls, and enterprise support through AMP. A team can build its own operating layer, but it should not read the open-source quick start as proof that production access control and incident handling are already solved.

Active releases do not shrink an 806-item work queue

GitHub showed 57,605 stars and 806 open issues and pull requests combined when we fetched the repository. The last push was August 25, 2026, and release 1.15.17 arrived on August 20. Recent activity included fixes and discussions around async model-call hooks, optional fields in task output schemas, and pre-tool-call authorization. That is a busy maintained project, but the combined queue is large enough that adopters should pin a version and test the provider and tool paths they actually use.

CrewAI is easiest to justify when the domain already maps to roles and handoffs, such as research followed by review, and when a Flow can contain the uncertain parts. For a short deterministic pipeline, direct SDK calls will be easier to debug. For a multi-agent product, CrewAI offers a readable vocabulary and a large community, with a contributor setup that our 29-second install did not fully validate.

Alternatives

ProjectWhat it isPick it when
LangGraph gh↗A graph-based runtime for stateful agents with explicit nodes, edges, and persistence.pick this instead when workflow state and graph control matter more than role-playing agent teams.
Microsoft Agent Framework gh↗Microsoft's framework for composing agents and workflows across Python and .NET.pick this instead when your organization needs .NET support or Microsoft-aligned agent infrastructure.
AutoGen gh↗A multi-agent programming framework centered on agent conversations and event-driven applications.pick this instead when conversational agent patterns and its existing extension ecosystem fit your design better.

Sources

  1. CrewAI README
  2. CrewAI 1.15.17 release
  3. CrewAI optional output fields issue
  4. CrewAI async hook issue

More llm tools reviews

headcount · useagent · claude-skills · RAG_Techniques · ux-ui-agent-skills · dictionary-of-ai-coding · the whole board →