mrkeyoor.com_
Sat 31 Oct 15:03 UTC
AI Toolsevaluationupdated 01 Oct 2026

firecrawl review

Firecrawl is a web data API that searches, scrapes, maps, crawls, and interacts with pages, then returns markdown, structured data, screenshots, or metadata. It handles the browser, queue, and content-cleaning work between an agent asking for web information and an application receiving usable results.

+1,777stars / 7d
Verdict

Our Firecrawl run installed 778 packages in 87 seconds, then failed its build and ended with 170 of 2,463 tests failing, so the source checkout did not clear a clean release gate. Start with the hosted API when one interface for search, scraping, crawling, and browser actions would replace several services. Self-host only when data or deployment policy justifies owning the workers, queues, databases, browser service, security, and upgrade path.

We ran it

Lab card: what happened when we ran firecrawlScreenshot of firecrawl (firecrawl.dev)
Install✓ · 87s778 packages · 890 MB
Build✗ · 11s
Tests✗ · 116s2228 passed · 170 failed · 65 skipped of 2463 (vitest)
Repo1975 files~360,445 lines of source · 53.3 MB · 31 CI workflows · Dockerfile

Answers from our run

Does firecrawl build from source?

Dependencies installed in 87 seconds (778 packages), and the build failed. We cloned commit b5d187b into a clean Debian container with 3 CPUs and no project-specific setup.

Do firecrawl's tests pass?

No. The project's own test command finished in 116 seconds and reported 2228 passed, 170 failed, 65 skipped (vitest).

Who should not use firecrawl?

Self-hosters expecting the cloud product in one free container: the README says cloud has additional features, while the source guide runs API workers, Playwright, Redis, RabbitMQ, and PostgreSQL-backed queues.

What are the alternatives to firecrawl?

Crawl4AI, Crawlee, Scrapy. Our Firecrawl run installed 778 packages in 87 seconds, then failed its build and ended with 170 of 2,463 tests failing, so the source checkout did not clear a clean release gate.

Setup2/587-second install, followed by a failed build and 170 failed tests
Docs4/5Clear API examples and candid self-hosting responsibilities
Community5/5187,301 stars with an October 1 push and active issue work
Maturity3/5Wide API and SDK coverage, but our source checks failed

Discussed on

  1. hnShow HN: Firecrawl-Simple – Stable fork of Firecrawl optimized for self-hosting35 points
  2. hnShow HN: Draco – A single-binary, self-hostable Firecrawl alternative in Rust15 points
  3. hnPdf-inspector: Rust lib for PDF inspection, classification, and text extraction5 points
  4. hnFast Rust Library for PDF text extraction5 points
  5. hnFirecrawl: Turn entire websites into LLM-ready Markdown or structured data4 points

Who it’s for

Product teams that need search, scraping, crawling, and browser interaction behind one API.
Agent and RAG builders who want markdown or structured data instead of raw page HTML.
Teams that prefer to test a hosted API before deciding whether web extraction deserves its own platform.
Platform operators who need a self-hosted web data service and can own its queues, browsers, databases, and worker capacity.
MCP and Claude Code users who want web retrieval available inside an agent workflow.

Who it’s NOT for

Self-hosters expecting the cloud product in one free container: the README says cloud has additional features, while the source guide runs API workers, Playwright, Redis, RabbitMQ, and PostgreSQL-backed queues.
Teams requiring a clean source gate before evaluation: our build failed, and 170 of 2,463 tests failed at commit b5d187b.
Operators who would expose the default Compose setup publicly: the guide starts with database authentication disabled and assigns TLS, network policy, persistence, and backups to the operator.
Evidence-sensitive extraction systems without their own checks: open issue 4653 reports silent local-model truncation, while issue 4773 reports that Playwright can omit the final navigation URL.
Closed-source network services whose owners have not reviewed AGPL-3.0 obligations; the repository says only SDKs and some UI components use MIT terms.

Setup reality

Our sandbox installed commit b5d187b from apps/api in 87 seconds: 778 pnpm packages and 890 MB on disk. The 53.3 MB checkout held 1,975 files and about 360,445 source lines. The build failed with exit 2 after 11 seconds. Vitest exited 1 after 116 seconds: 2,228 passed, 170 failed, and 65 skipped out of 2,463.

Hosted use needs a Firecrawl API key. Self-hosting adds API workers, Playwright, Redis, RabbitMQ, NuQ PostgreSQL, and optional FoundationDB. AI-backed features need OpenAI, an OpenAI-compatible endpoint, or Ollama. The default Compose path publishes port 3002 and begins with database authentication disabled.

The build log treated pdfResult as unknown and could not resolve @mendable/firecrawl-rs in Playwright and extraction helpers. The test tail showed failing authenticated v0 route cases. The log does not establish a shared cause. The monorepo has 31 CI workflows and a Dockerfile, but our clean Debian run did not produce a passing build or suite.

Seven web jobs sit behind one API

Firecrawl's v2 interface covers search, scraping, interaction, agents, crawling, site mapping, and batch scraping. A caller can request markdown, HTML, structured JSON, screenshots, links, images, or metadata. That is a useful boundary when an agent must find a source, render JavaScript, click into the right state, and return cleaned content without the product team assembling a search provider, browser farm, queue, and HTML cleaner.

The hosted quick start asks for an API key and works through Python, Node.js, curl, or the CLI. The README also lists Go, Java, Elixir, Rust, Ruby, .NET, and PHP SDKs. Firecrawl supplies an MCP server and a skill that works with Claude Code. The breadth is convenient, but each extra format and interaction becomes another behavior to validate against the sites your product depends on.

What happened when we ran it

Our sandbox installed commit b5d187b from apps/api in 87 seconds. Pnpm pulled 778 packages and occupied 890 MB. The checkout contained 1,975 files, roughly 360,445 lines of source, and used 53.3 MB before dependencies. This was a fresh Debian container with 3 CPUs, 8 GB of RAM, Node 22, no secrets, and no elevated privileges. Installation was the only step that passed.

The build exited 2 after 11 seconds. TypeScript reported several PDF-engine lines where pdfResult had type unknown. It also could not find @mendable/firecrawl-rs or its declarations in the Playwright engine and helpers for attributes, images, links, metadata, and unwanted elements. The log shows those compiler failures. It does not tell us whether another generation step, package build, or dependency arrangement would resolve them.

Vitest exited 1 after 116 seconds. It counted 2,228 passing tests, 170 failures, and 65 skipped tests out of 2,463. The file summary was 158 failed, 160 passed, and 6 skipped across 324 files. The final lines named authenticated v0 end-to-end cases around search, crawl status, cancellation, and LLM extraction. Those names identify the affected routes, but the tail does not prove why all 170 failed.

The repository has 31 CI workflow files, a Dockerfile, and monorepo workspaces. Our scan found no top-level tests directory, although the Vitest run clearly found suites under apps/api. A resolved 778-package graph is useful. It is still weaker evidence than a compiler and test suite that finish cleanly, and commit b5d187b did not meet that bar in our sandbox.

Port 3002 fronts a stateful self-hosted stack

The source-aligned guide says Compose publishes the API on port 3002. Behind it run Firecrawl API workers, Playwright, Redis, RabbitMQ, and NuQ PostgreSQL. FoundationDB is available as an optional queue backend. AI-backed features need OpenAI, an OpenAI-compatible endpoint, or Ollama. This is a platform with several failure and persistence boundaries, rather than a scraper packed into one process.

The first-run advice keeps USE_DB_AUTHENTICATION=false until the required schema and application configuration exist. Firecrawl then places authentication, TLS termination, network policy, backups, recovery tests, monitoring, limits, scaling, and rollback on the operator. The checked-in Compose file defines no persistent volumes for PostgreSQL, Redis, or RabbitMQ. That is honest documentation and a warning against treating docker compose up as a finished public deployment.

Open issue 4653 shows why JSON needs source checks

Issue 4653 reports that self-hosted JSON extraction can send a full page to a local model with an 8k to 32k token context. In the supplied reproduction, Ollama truncated the prompt and returned confident wrong fields without warning the caller. This is a user report, separate from our 2,463-test run. It still describes a serious failure mode for anyone storing extracted facts without the source text that produced them.

Issue 4773 documents another evidence gap: the self-hosted Playwright service followed a redirect and returned the destination content, but omitted the final observed URL. Issue 4348 reports an anti-bot waterfall that can repeat until timeout on a blocked page. All 3 issues were open when checked. Retain requested and returned URLs, bound job time, reject implausible empty output, and compare important structured fields with page evidence.

AGPL-3.0 covers the server, while SDK terms differ

The repository is primarily AGPL-3.0. Its README says SDKs and some interface components use MIT licenses, with directory-level license files defining the details. That split matters if you modify the server and make it available over a network. A commercial team should review the exact directories it will deploy and understand the corresponding source-sharing obligations before choosing self-hosting.

The hosted and self-hosted products are not identical. Firecrawl says the cloud edition includes additional features, while the source guide gives operators control over infrastructure and data. Hosted use shifts browsers, proxy operations, queues, availability, and upgrades to Firecrawl. Self-hosting gives you more control but returns every one of those responsibilities, plus compliance with websites' policies and the project's license.

187,301 stars accompany same-day code activity

GitHub showed 187,301 stars and a push on October 1, 2026. Its combined open counter was 505; separate searches split that into 89 issues and 416 pull requests, so 505 is not a bug count. The latest tagged release was v2.11.0 from June 19, while the October push and recent issue updates show work continued after that tag. Pin the exact source or image you test instead of reading release age as abandonment.

Firecrawl is easiest to recommend as a hosted boundary around an unpleasant engineering problem. Test it on the JavaScript-heavy, redirected, blocked, and long pages your product actually needs. The source edition deserves a platform owner, an AGPL review, and a repeat of the build and 2,463-test gate on the chosen revision. Our failed run makes that proof a requirement, not a paperwork step.

Alternatives

ProjectWhat it isPick it when
Crawl4AI gh↗A Python crawler that turns websites into content shaped for AI applications.pick this instead when you want a Python-first local crawler and will build the surrounding service yourself.
Crawlee gh↗A JavaScript and TypeScript library for browser automation and custom crawlers.pick this instead when crawler behavior belongs in your application code rather than behind a web data API.
Scrapy gh↗A Python framework for custom crawling, extraction pipelines, and request scheduling.pick this instead when precise crawler logic and item pipelines matter more than agent-ready markdown.

What people are saying

  1. [github-trending] firecrawl/firecrawl
  2. [producthunt] Alexandria by Firecrawl
  3. [github-trending] firecrawl/pdf-inspector

Sources

  1. Firecrawl README
  2. Firecrawl source-aligned self-hosting guide
  3. Firecrawl contributing guide
  4. Firecrawl v2.11.0 release
  5. Issue 4653: local-model extraction truncation
  6. Issue 4773: Playwright omits final navigation URL
  7. Issue 4348: anti-bot waterfall can loop until timeout

More ai tools reviews

KiroCrew · Orkas · hypit · Paradev · see-through · Resume-Matcher · the whole board →

Related reading