One API spans requests, browsers, parsing, and spiders
Scrapling starts as a page fetcher and parser, then extends into a crawler framework. Fetcher handles ordinary HTTP work, DynamicFetcher drives Chromium or Chrome, and StealthyFetcher targets sites with anti-bot checks. Responses support CSS, XPath, text, regular expression, and traversal methods. The spider layer adds concurrency, sessions, request scheduling, pause and resume, proxy rotation, throttling, streaming output, and exporters. This consolidation is useful when a scraping job keeps outgrowing its first script.
The project is compact compared with that feature list. Our checkout contained 254 files, about 28,220 source lines, and occupied 4.4 MB before dependencies. GitHub showed 76,676 stars, 0 open issues and pull requests, and a last push on August 25, 2026. Release v0.4.15 arrived August 23. Zero open items is a snapshot of the public queue, not proof that every target site or optional installation combination works.
Adaptive selectors trade brittleness for possible ambiguity
Scrapling can save information about a selected element and later relocate it when a page structure changes. That can keep an extractor alive after class names or nesting move. The cost is that a similarity match may select the wrong element while still returning plausible data. Use adaptive lookup as recovery, then validate fields with types, ranges, required labels, and record counts. A scraper that never crashes can be more dangerous than one that fails loudly if it silently shifts to a nearby price or title.
The framework also supplies ready-made spider templates for rule-based links, sitemaps, XML or CSV feeds, Shopify data, and site-to-Markdown jobs. Development mode caches responses so parser changes do not keep hitting the target. Those are practical features for repeated work. They do not remove target-specific tests. Save representative pages, including blocked and malformed responses, and run extraction assertions before deploying a selector change across 10,000 URLs.
What happened when we ran it
Our sandbox cloned commit 4ce0fb1 into a fresh unprivileged Debian container with 3 CPUs and 8 GB of RAM. Installation succeeded in 24 seconds, adding 39 Python packages and consuming 49 MB. The build succeeded in 14 seconds. Pip-audit reported 0 known vulnerabilities in the installed Python environment.
Tests failed with exit code 1 after 80 seconds. Pytest reported 258 passed, 5 failed, and 110 collection or setup errors out of 373. The final lines listed errors in spider templates, throttling, and shell functionality. Seven shown CLI parser cases raised ModuleNotFoundError: No module named 'curl_cffi', including basic curl parsing, headers, data, cookies, proxies, conversion to a fetcher, and invalid commands.
The log tail establishes one missing module in those errors, but it does not show that all 110 errors share that cause or why the package was absent. We therefore treat the run as mixed: the 24-second install and 14-second build passed, 258 tests worked, and the full suite did not. Anyone adopting optional shell, browser, spider, or AI features should install the matching extras and run the relevant suites in the intended image.
Browser and stealth modes carry the operating cost
Plain HTTP scraping is cheap. Browser sessions add executable downloads, memory, process cleanup, page timeouts, and behavior that changes with browser versions. Scrapling can keep tabs open across requests, connect to remote browsers over CDP, capture matching XHR responses, block resource domains, and use DNS over HTTPS. Each feature solves a real class of target, but each also expands the failure surface beyond a normal Python request.
Release v0.4.15 changed tab reuse so sessions keep pages open and apply per-request settings to the reused tab. It also fixed error replacement and added close_pages(). Long-running workers should still expose browser counts, open pages, request failures, memory, and shutdown success. A leaked tab repeated across 500 requests becomes an operational problem. Use the cheapest fetcher that reliably obtains the data, and reserve browsers for pages that truly require them.
MCP access should stay local or authenticated
Scrapling's MCP server lets agents perform plain, stealth, and browser fetches, maintain sessions, capture screenshots, and narrow pages with selectors. The project says it strips scripts, styles, and hidden prompt-injection content before returning page material. That cleaning is useful, but fetched text can still contain misleading instructions in visible content. An agent should treat page text as data rather than authority.
Version 0.4.15 reworked the server's 13 tools into one-shot and session modes, renamed get to make_request, and added persistent HTTP sessions. The release labels these as breaking changes. HTTP transport now binds to localhost and requires a bearer token by default; network exposure needs an explicit host setting. Keep that default, rotate the token, restrict reachable destinations, and review upgrades before changing an agent configuration.
Scraping claims need legal and data-quality checks
The README says stealth fetchers can handle Cloudflare Turnstile and advertises browser fingerprint impersonation, proxies, and blocked-request retries. Technical capability is not authorization. Operators need a documented purpose, target terms review, privacy controls, respectful request rates, and a deletion policy. The optional robots setting can honor Disallow, Crawl-delay, and Request-rate; decide policy centrally instead of leaving each script author to toggle it.
Scrapling earns a trial when a team wants one Python interface from a single fetch through a resumable crawl or MCP tool. Its 49 MB installed footprint is modest, while the failed optional-feature test setup shows why extras must be pinned and checked. For stable sites, simpler tools remain easier to trust. For difficult targets, Scrapling offers useful control, provided the team validates extracted data and owns the browser, proxy, policy, and upgrade work.