One API between agents and the messy web
Firecrawl handles several jobs that teams often assemble separately. It can search, scrape one page, map a site's URLs, crawl many pages, batch a list, and operate a browser after a scrape. Results can include markdown, HTML, screenshots, structured JSON, links, images, and other page data. SDKs cover multiple languages, while the CLI, MCP server, and agent skill put the same service into interactive workflows.
The appeal is consistency. An agent can ask for search results with page content, then send a promising URL through the same client for cleaner extraction. A crawler can discover links, apply path limits, and return asynchronous job status rather than forcing application code to coordinate every browser. Browser interaction covers pages that need clicks, text entry, waits, or scrolling before the useful state appears.
Hosted Firecrawl also owns proxying, browser capacity, rate limits, and worker orchestration. That is the product people are usually buying. The repository makes the API inspectable and offers a self-host path, but the README is candid that cloud includes additional features. Compare editions before choosing self-hosting for cost alone.
What happened when we ran it
We cloned commit 9835405 into a fresh unprivileged Debian container with 3 CPUs, 8 GB of RAM, and no secrets. The repository held 1,764 files, about 304,255 lines of source, and occupied 51.3 MB. Its Node and pnpm project lives under apps/api. Installation succeeded in 66 seconds, bringing in 727 packages and using 692 MB.
The build exited 2 after 9 seconds. TypeScript could not find @mendable/firecrawl-rs or its declarations in sitemap code and several scraper paths, including document, PDF, Playwright, metadata, link, image, and element helpers. The log establishes a missing module during compilation. It does not tell us whether the package should have been generated, downloaded, or built through another documented path.
Vitest ran for 256 seconds and exited 1. Of 2,166 tests, 1,930 passed, 183 failed, and 53 were skipped. The summary reports 90 failed test files, 138 passed files, and five skipped files. The last failures concern authenticated v0 E2E routes for search, crawl status, cancellation, and LLM extraction. The tail ends with one test timing out after 120 seconds. We cannot assign a common cause from those lines.
The measured checkout has 31 CI workflow files, a Dockerfile, and monorepo workspaces, but no top-level tests directory. The high passing count shows substantial coverage executed. The build error and failed E2E group still mean this exact clean checkout did not clear its development gates.
Hosted use is the easy route
The quick start asks for a Firecrawl account and API key, then shows the same operations in Python, JavaScript, curl, and the CLI. A single scrape is short enough to understand immediately. Search can return page content with each result, and crawl or batch work moves into job APIs. This is the right entry point for a team evaluating extraction quality rather than infrastructure.
Agent integration is also direct. The README provides an MCP configuration and an install command for its agent skill, with Claude Code and OpenCode named as supported clients. MCP saves adapter work, but it can expose web access broadly to the model. Scope credentials, limit which tools are enabled, and keep application-side rules around allowed targets.
Firecrawl's feature surface is wider than plain HTML cleanup. The June 2026 v2.11.0 release added PII redaction, deterministic JSON extraction, broader video discovery, CDP access to browser sessions, and changes to monitoring. Some of those capabilities depend on the hosted product or external model services, so an API example should not be read as proof that every feature appears in the community deployment.
Self-hosting means owning the workers
The contribution guide separates development from deployment and warns against copying their environment files between paths. API development calls for Node.js 22, pnpm 11.4.0, a separately running Redis, and a harness that manages PostgreSQL and RabbitMQ containers. The start command builds the API and launches workers plus local dependencies. This is already more operational surface than a scraper package.
Production adds browser concurrency, queue recovery, network egress, secrets, storage, and observability. Search and LLM-based JSON extraction bring extra providers and their failure modes. AGPL-3.0 is the main repository license, while SDKs and some UI components use MIT. A company serving a modified version over a network should get a license review before deployment.
Recent issues give useful warnings for API consumers. Issue #4393 says onlyCleanContent exists in server schemas but was missing from nine SDKs and could be ignored silently by the Python model. Issue #4375 shows a malformed DuckDuckGo result URL crashing self-hosted search while the endpoint responds with an empty successful result. Issue #4374 reports default SDK pagination blocking a crawl-status call while the crawl keeps growing. Validate requested output fields, set timeouts, and distinguish empty data from an operational success signal.
Health and the decision
The repository now lives under the firecrawl organization, with GitHub redirecting the old mendableai path. The last push was August 24, 2026. Issues were updated on August 25, and GitHub reported 553 open issues and pull requests combined. Release v2.11.0 arrived June 19 after monthly listed releases in April and May. Both code and issue activity are current.
Firecrawl makes the most sense as a service boundary: applications call a stable scraping API while a specialist team handles browsers and evasive pages. The hosted edition is the practical first test. Self-host when policy, data location, or deep customization outweighs the burden, and require a clean build plus relevant E2E checks before promoting a source revision.