mrkeyoor.com_
Mon 10 Aug 17:49 UTC
Self-Hostedevaluationupdated 10 Aug 2026

anubis

Anubis is a self-hosted filter that sits in front of a website and makes suspicious visitors perform a small proof-of-work challenge before reaching it. It aims to make large-scale AI scraping expensive while letting ordinary browser users through with a temporary cookie.

Verdict

Anubis is a strong self-hosted answer to crawler load when a managed CDN is unacceptable and occasional visitor friction is an acceptable price. It is not a universal bot detector, and its defaults are a starting point rather than a production policy. Deploy it first on one noncritical service, watch pass and block rates, explicitly preserve useful robots and clients, and keep a bypass path for mistakes.

Setup3/5Simple container, substantial proxy, state, cookie, and policy work
Docs5/5Detailed environments, policy language, storage, and caveats
Community5/5Large adoption with frequent releases and daily issue activity
Maturity4/5Widely deployed v1 project, still correcting upgrade edge cases

Who it’s for

  • Small website and community operators overwhelmed by automated scraping traffic.
  • Self-hosters who cannot or do not want to place a site behind Cloudflare.
  • Nginx, Caddy, Apache, HAProxy, Traefik, or Kubernetes operators comfortable inserting another reverse-proxy hop.
  • Administrators willing to tune allow, deny, challenge, and request-weight policies from real traffic.
  • Sites that can tolerate an occasional browser challenge in exchange for reducing bulk crawler access.

Who it’s NOT for

  • Publishers that depend on broad archival, search, RSS, or small-browser access without maintaining allowlists: the README calls Anubis a nuclear response that can block smaller scrapers and inhibit good bots such as the Internet Archive.
  • WebSocket-heavy applications: the installation guide says Anubis may be a poor fit and notes limited real-world experience with long-lived connections.
  • Sites where every visitor must work without JavaScript, cookies, or client computation: proof-of-work and a pass cookie are central to the normal challenge flow.
  • Operators seeking certain human-versus-bot identification: the design documentation calls proof-of-work a good-enough placeholder, and says some bots can pass challenges.
  • Anyone unwilling to own reverse-proxy headers and cookie details: the X-Forwarded-For caveat warns that a wrong public-proxy chain can block everyone or cause an infinite challenge loop.

Setup reality

The container itself is small, but deployment is not one command in front of a real site. Anubis belongs between TLS termination and the upstream service, so you must change proxy routing, preserve the real client IP, select and tune a policy, set cookie behavior, protect metrics, and test every non-browser client. Production also needs bbolt, Valkey, or another supported persistent store because the default memory backend is explicitly discouraged, plus a stable signing secret so challenges survive restarts. Multi-instance setups need shared state and keys.

A speed bump for industrial scraping

Anubis is a reverse-proxy filter built around a simple economic idea. A normal visitor loads a page occasionally, so asking that browser to calculate a SHA-256 proof-of-work result costs little in aggregate. A scraper requesting pages at industrial scale must repeat the work constantly. The server verifies a successful nonce with one hash, then gives the visitor a temporary pass cookie.

This does not prove that a visitor is human. The project says some bots can solve the challenge and describes proof-of-work as a good-enough placeholder while fingerprinting improves. What it does is shift cost toward bulk clients and protect an upstream site from immediately processing every request. That can be valuable for a small forge, documentation site, archive, or community server whose capacity is being consumed by AI crawlers.

Anubis is also unusually candid about when not to use it. Its README calls the approach a nuclear response, warns that smaller and good crawlers may be blocked, and says most sites can probably use Cloudflare instead. Self-hosting is the reason to choose Anubis, not an assumption that it provides better identification than a large managed edge.

Policies matter more than challenge difficulty

Anubis can allow, deny, challenge, or adjust the suspicion weight of a request. Rules can match user agents, paths, remote address ranges, headers, and expressions. Standard policy snippets cover known crawlers and clients, while imports let administrators compose a local policy without copying every upstream rule. Prometheus metrics expose named-rule outcomes, which is necessary for tuning.

The default posture is intentionally aggressive. Documentation says browser-like clients, usually those with Mozilla in the user agent, are challenged. It also notes that RSS readers and other bot-looking clients may be legitimate. Small browsers such as Dillo, NetSurf, and Pale Moon need an optional standard rule to bypass expensive challenges. Good search and archive bots may need verified address rules.

That means the main operating job is classification, not picking a larger difficulty number. A broad catch-all rule can create false positives, punish assistive or low-power clients, and damage discovery. Raising proof-of-work cost also taxes legitimate visitors. Start with the supplied policies, record which rules fire, and add narrow exceptions from observed traffic. Anubis should reduce abusive load without becoming the main reason users cannot reach the site.

The proxy diagram hides important work

The expected path is load balancer or TLS terminator, then Anubis, then the application. Docker Compose can start an Anubis container beside Nginx, and the project documents Nginx, Caddy, Apache, HAProxy, Traefik, Kubernetes, native packages, and beta Windows Server support. The Go service itself has modest stated requirements.

A dependable deployment still requires correct routing in both directions. The proxy must pass the real client address and prevent users from spoofing trusted headers. Anubis documents a specific X-Forwarded-For failure: with a public proxy such as Cloudflare in the chain, it can select the proxy address instead of the visitor. The result may be universal blocking or an endless challenge loop. Administrators may need to normalize the header or configure a dedicated real-IP header.

Cookies add another layer. Secure, SameSite, partitioned, domain, and expiration settings must match the site's HTTPS and embedding behavior. Release 1.27.0 began deriving cookie names from settings because browsers can retain same-named cookies with different attributes, previously causing infinite challenge loops after a configuration change. Open issue #1851 says the documented COOKIE_PARTITIONED default lagged the release and describes Firefox rejecting cookies after an upgrade when combined with insecure-cookie settings. Test upgrades with old browser state, not only clean sessions.

Production needs state and escape routes

The default in-memory store makes a demo easy, but the policy documentation says not to use it persistently in production and notes that it currently has no size limit. A single host can use bbolt, which takes an exclusive database lock. Multiple replicas can share Valkey, S3-compatible object storage, or another documented network backend, each with its own request and operations cost. Persistent stores require a stable Ed25519 or HS512 secret so outstanding challenges survive restarts, and every instance on the same base domain must agree on that key.

Plan a bypass before going live. Health checks, monitoring systems, APIs, Git clients, mobile applications, and webhooks cannot be assumed to complete a browser challenge. Policies should explicitly allow or authenticate those paths and clients. A direct administrative route can keep an application reachable when a cookie, header, or rule change locks out normal traffic.

WebSockets deserve separate testing. The installation guide says Anubis may be a poor fit for applications with long-lived WebSocket connections and acknowledges limited experience. Open issue #1847 also reports browsers occasionally getting stuck after solving a challenge, possibly around service restarts. That report is not proof of a general outage pattern, but it reinforces the need for realistic session and restart tests.

Fast-moving and well documented

The repository was pushed on August 10, 2026, two days after version 1.27.0. The combined open count of 346 includes issues and pull requests. Recent work covers crawler ranges, cookie behavior, proxy configurations, translations, dependency updates, and bug reports, so the large queue sits alongside very active maintenance rather than a stale codebase. More than 21,000 stars and packaging across several ecosystems show considerable attention for a project created in 2025.

Documentation is a major strength. It goes well beyond a container example into proxy-specific layouts, a policy language, storage choices, metrics protection, header caveats, cookies, bot lists, and design rationale. The project also labels Windows Server support as beta and admits uncertainty around WebSockets instead of projecting certainty.

Choose Anubis when scraper pressure threatens availability and retaining infrastructure control matters more than invisible access. Treat it as a tunable traffic filter, not a truth machine. The best deployment is not the one with the hardest challenge; it is the one that measurably lowers waste while legitimate readers, tools, archives, and integrations continue to work.

Alternatives

ProjectWhat it isPick it when
Cloudflare Bot ManagementA managed edge service that combines bot signals, challenges, rate controls, and global proxying.pick this instead when you accept a hosted intermediary and want less proxy, policy, and crawler maintenance.
CrowdSecA behavior-based security engine that detects abusive clients and feeds decisions to firewalls and proxies.pick this instead when you want shared reputation and behavior-based blocking across services rather than browser proof-of-work.
OWASP Core Rule SetA community rule set for detecting common web attacks in compatible web application firewalls.pick this instead when exploit payloads and application attacks are the main threat, not high-volume scraping.

What people are saying

  1. [github-trending] TecharoHQ/anubis

Sources

  1. Anubis README
  2. Anubis installation guide
  3. Anubis policy documentation
  4. Why Anubis uses proof-of-work
  5. Anubis 1.27.0 release
  6. Cookie default documentation issue
  7. Challenge page stuck report