Anubis trades universal access for lower scraper load
Anubis sits between a visitor and an existing web application. A request that policy does not trust receives a JavaScript challenge, and successful clients get a signed cookie before traffic reaches the origin. The goal is to make bulk scraping consume enough client work that a small server stays usable. It is an HTTP gate, not a crawler detector that can identify every company or intent with certainty.
The README is candid about the trade. It calls Anubis a nuclear response and says smaller scrapers or useful bots such as the Internet Archive may be blocked. Administrators can write policy rules and allow known crawlers, but that makes bot classification an operating task. A site serving researchers, feed readers, command-line clients, or unusual browsers should inventory those users before turning the challenge on.
The 24-second install failed before compilation began
We cloned commit 2f45638 into an unprivileged Debian container with 3 CPUs and 8 GB of RAM using the lab Node 22 image. The checkout contained 785 files, about 31,961 source lines, and occupied 4.9 MB. Npm began installation, ran the package's prepare script, invoked Husky, and then called go mod download.
That Go command rejected the module file. The log says line 3 contains an invalid Go version, 1.26.3, for the parser's expected format, and line 204 has an unknown tool block. Npm exited with code 1 after 24 seconds. The failure shows that this source checkout expects a newer or different Go toolchain than the one available in that sandbox. The log does not show a missing network dependency or application defect.
What happened when we ran it
Our sandbox never completed installation. The prepare hook failed after 24 seconds at go mod download, so the harness had no valid dependency state from which to build or test. We did not run a binary, solve a browser challenge, or send traffic through a protected origin. Any claim about request capacity, proof cost, or crawler reduction would go beyond this measurement.
The repository scan found 18 CI workflow files and a tests directory, but no Dockerfile. That does not mean containers are unavailable: the project publishes images and the release notes discuss container tags. It means our source scan and npm path are different from the packaged deployment route an administrator would normally follow. Source contributors should match the repository's Go declaration before interpreting any later compiler or test result.
Reverse-proxy correctness decides whether users return to the right page
Anubis needs an upstream target, public URL, signing key, policy file, cookie configuration, and a listener behind the site's ingress. The ingress must preserve the original host, path, scheme, and client address correctly. Getting one header wrong can affect policy decisions or the post-challenge redirect. Metrics should be bound where the monitoring system can reach them without exposing operational data to the public internet.
Open issue #1886 reports that a Forgejo request for /foo returned to the site homepage after the user solved the challenge. The reporter supplied Caddy headers and a Compose configuration, and Anubis logged the original path. Issue #1878 describes malformed redirects for a path without a trailing slash in an Apache setup. These are individual reports, not proof that every proxy combination fails. They justify explicit tests for deep links, query strings, trailing slashes, and login callbacks.
Browser challenges exclude more than hostile AI crawlers
A normal graphical browser can run the challenge and retain its cookie. Text browsers, scripts, monitoring probes, RSS readers, package clients, and users who disable JavaScript may not. Search and archive crawlers need deliberate policy decisions. Release v1.27.0 added rules for Arquivo.pt, Dillo, NetSurf, Pale Moon, Googlebot addresses, DuckDuckBot, and another AI crawler, showing how much maintenance sits around the challenge itself.
Local services are another edge case. Issue #1888 asks Anubis to allow localhost by default because same-machine clients otherwise face the gate. The reporter also notes that such a default would make local reproduction harder, so they plan to handle it in user policy. That is the correct mental model: an allow rule changes both access and what can be tested from a host. Monitor checks should exercise the protected public route as well as the origin.
Active releases do not make a blanket policy safe
Version 1.27.0 was published on August 8, 2026. It introduced beta Windows Server packages, changed cookie names based on settings, and fixed a release process that had allowed prerelease containers to populate the latest tag. GitHub showed 21,732 stars, 364 combined issues and pull requests, and an August 25 push. Those facts show active maintenance and a sizable user queue.
The dynamic cookie change fixes loops caused by old cookies surviving configuration changes, but it can affect advanced setups that inspect cookie names. The earlier container-tag mistake is also a reason to pin releases rather than follow latest. Anubis is a sensible response when logs prove that scraper load threatens uptime. If Cloudflare is already acceptable, or if non-JavaScript access is part of the service contract, its own README points toward a less disruptive choice.