mrkeyoor.com_
Sat 31 Oct 07:47 UTC
LLM Toolsevaluationupdated 15 Oct 2026

crewAI review

CrewAI is a Python framework for giving AI workers separate jobs and passing work between them. It also lets you put those workers inside workflows whose branching and state you control in code.

+218stars / 7d
Verdict

Our CrewAI run installed 34 packages in 19 seconds, then failed its build and stopped its test step on a missing dotenv import. Use it for Python workflows whose separate agent roles justify the coordination layer, provided you can establish a reproducible environment first. I'd require complete MCP discovery checks for the intended server and review the research passed between agents before trusting a finished report.

We ran it

Lab card: what happened when we ran crewAIScreenshot of crewAI (crewai.com)
Install✓ · 19s34 packages · 37 MB
Build✗ · 9s
Tests✗ · 8sran, no count parsed
Known vulns0(pip-audit)
Repo34979 files~362,104 lines of source · 443.2 MB · 16 CI workflows

Answers from our run

Does crewAI build from source?

Dependencies installed in 19 seconds (34 packages), and the build failed. We cloned commit 75808b0 into a clean Debian container with 3 CPUs and no project-specific setup.

Do crewAI's tests pass?

No. The project's own test command exited unsuccessfully after 8 seconds, and its output did not expose an individual test count.

Does crewAI have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use crewAI?

Teams requiring a clean source build and test run before evaluation: our build failed, and the test step stopped on a missing dotenv import.

What are the alternatives to crewAI?

LangGraph, Pydantic AI. Our CrewAI run installed 34 packages in 19 seconds, then failed its build and stopped its test step on a missing dotenv import.

Setup2/519-second install succeeded, but build and test steps failed
Docs4/5Detailed Flow and setup guides, with separate CLI and project versions
Community5/5October 16 push and current issue and pull-request activity
Maturity3/5Active 1.15.28 release, failed lab checks, and open MCP discovery report

Who it’s for

Python developers splitting research and reporting into tasks with explicit owners and expected outputs.
Teams that want ordinary Python workflow steps around agent collaboration, using Flows for branching and state.
Engineers prepared to configure model providers, tool credentials, and checks on intermediate agent output.

Who it’s NOT for

Teams requiring a clean source build and test run before evaluation: our build failed, and the test step stopped on a missing dotenv import.
Applications that need complete discovery from paginated MCP servers without checking compatibility: open issue #8216 reports missing prompts and HTTPS-discovered tools in 1.15.28.
Operators who cannot accept default usage telemetry without configuring an opt-out: agent roles and tool names are collected, and share_crew expands collection.
Projects standardized on Python 3.14: the documented supported range stops below it.

Setup reality

Our run at commit 75808b0 installed 34 packages in 19 seconds and occupied 37 MB. The build failed with exit 1 after 9 seconds. Tests failed with exit 4 after 8 seconds: conftest.py imported dotenv, which was unavailable. The build's cause is absent from the supplied log. Pip-audit reported 0 known vulnerabilities.

The README's research example needs a model-provider credential and a Serper search key. Local model connections are documented. Configure agents and tasks in JSONC, or choose the classic Python/YAML scaffold. The global CLI and your project's environment need separate version management.

Python must be at least 3.10 and below 3.14. Our fresh Debian sandbox used Python 3.12, 3 CPUs, 8 GB of RAM, and no secrets. That supported interpreter did not produce a clean build or test run. Set an explicit telemetry policy before processing internal material.

A researcher can pass its findings to a reporting agent

CrewAI's README gives a researcher a search tool and passes its findings to a reporting agent, which writes a Markdown file. That handoff is a good reason to consider this framework. Each job has an owner and an expected result, and you can inspect the boundary between them. If your application needs those separate responsibilities, you can define those relationships in crew.jsonc. For a single extraction prompt, I'd start with a direct model call.

The source tree is substantial: our checkout contained 34,979 files, roughly 362,104 source lines, and occupied 443.2 MB. Those figures describe the repository, including material beyond your application. Our installed environment occupied 37 MB. Keep those measurements separate when deciding whether to contribute to the framework or use it as a library. Maintaining a source checkout and shipping a crew are different commitments.

What happened when we ran it

Our run installed CrewAI commit 75808b0 in 19 seconds, adding 34 packages and occupying 37 MB on disk. We ran it on October 14, 2026, in an unprivileged Debian container using ghcr.io/astral-sh/uv:python3.12-bookworm, with 3 CPUs, 8 GB of RAM, and no secrets. Installation succeeded. These measurements describe that sandbox run and commit, rather than every provider configuration the framework supports.

The build failed with exit 1 after 9 seconds. The supplied log tail does not explain the build failure. The test step failed with exit 4 after 8 seconds while loading /work/repo/conftest.py. Line 12 imports load_dotenv from dotenv, and Python raised ModuleNotFoundError: No module named 'dotenv'. That identifies the missing import. It does not explain why the module was unavailable, or establish what would happen after supplying it.

Pip-audit reported 0 known vulnerabilities in the environment it inspected. The repository scan found 16 CI workflow files, no Dockerfile, and no tests directory. Those are scan results, rather than proof that upstream has no tests or container deployment options. The immediate finding is narrower: installation completed, while the build and test steps failed. The supplied measurements contain no agent-quality benchmark or model-response timing.

Flows keep branching decisions in Python

CrewAI's Flows documentation shows typed state, conditional routing, and persisted state. A Flow can call ordinary Python code or invoke a crew at the step that needs one. For a research report, you could validate the input in code, ask a crew to gather material, and send its output for review. That is the architecture I'd start with because the approval decision remains visible in your program.

The agent API also exposes iteration limits and execution time limits. Use them to put bounds around delegated work, then define an acceptable output for each task. Our 8-second failed test step provides no evidence that a crew completes its assignment correctly. A reporting agent can receive another agent's mistakes along with its useful findings, so I'd inspect the intermediate research before trusting the final report.

The CLI and your project have separate environments

The installation guide uses uv tool install crewai for the global CLI. New crews use agents/*.jsonc and crew.jsonc; the --classic option produces the older Python/YAML layout. Upgrading the global CLI does not upgrade CrewAI inside an existing project's environment. Pin the project version and follow documentation matching it, especially when copying examples that expect a different scaffold.

Python must be at least 3.10 and below 3.14. Our Python 3.12 sandbox was inside that range, yet its build and tests still failed. A working interpreter is only part of setup. The README's research example also needs a model-provider credential and SERPER_API_KEY for search. Local model connections are documented, but choosing one still leaves you responsible for the model service and the tools your agents call.

Telemetry needs an explicit policy

CrewAI's telemetry documentation says default collection includes agent roles and tool names. Enabling share_crew expands collection to detailed crew and task data. CREWAI_DISABLE_TELEMETRY=true disables CrewAI telemetry, and OTEL_SDK_DISABLED=true is another documented option. For internal document work, set that policy before writing sensitive role names or enabling sharing. The MIT license permits code reuse, while the commercial AMP control plane is a separate offering.

An open MCP report describes missing discovery pages

Issue #8216 reports that CrewAI 1.15.28 returns only the first page of MCP prompts and misses later tools on its HTTPS discovery path. MCP lets agents reach tools exposed by compatible servers. The report names those specific paths and includes a reproduction; it is not a general claim that every MCP connection fails. Teams depending on a paginated tool catalog should verify discovery against their server before adopting this release.

October activity supports a trial, with a reproducibility condition

GitHub recorded a push on October 16, 2026, alongside recent issue and pull-request activity. Release 1.15.28 shipped on October 14 with fixes for resumed human feedback and deployment handling. The API listed 728 open issues and PRs combined. That is evidence of active maintenance and a busy queue, without establishing how quickly a particular report will be resolved.

Compare LangGraph when explicit graph state and durable execution are your starting requirements. Compare Pydantic AI when typed inputs, tools, and structured results matter more than CrewAI's role-and-task organization. CrewAI earns its place when the researcher-to-reporter handoff matches how you need to divide the work. Before building around that handoff, reproduce a clean build and test run for your pinned commit. Our 19-second install still left the build and test failures to resolve.

Alternatives

ProjectWhat it isPick it when
LangGraph gh↗A framework for stateful agent workflows with explicit graphs and durable execution.pick this instead when graph state and resumable execution are your preferred way to describe the workflow.
Pydantic AI gh↗A Python agent SDK built around typed tools, dependencies, and structured output.pick this instead when typed Python interfaces fit your application better than CrewAI's role-and-task organization.

Sources

  1. CrewAI README and repository
  2. CrewAI installation and project environments
  3. CrewAI Flows: state, routing, and persistence
  4. CrewAI telemetry collection and opt-outs
  5. Issue #8216: MCP prompt and HTTPS tool discovery pagination
  6. CrewAI release 1.15.28
  7. LangGraph repository
  8. Pydantic AI repository

More llm tools reviews

ironclaw · infinity · Claude-Code-Game-Studios · OmniRoute · harnessrouter · pi-subagents · the whole board →