Should you use Abide?
Use Abide for advisory feedback on repository rules that need judgment, provided someone will review its decisions. Abide lets edits continue when credentials or the network are unavailable. That behavior makes it unsuitable as the sole control for a requirement that must block changes.
Our run on October 6, 2026, at commit 0fc600f installed and built successfully, but the test command exited 1. For version 0.0.9, the evidence supports a limited trial alongside existing checks. It does not establish that a working agent session received and completed a repair.
What our sandbox established
Installation completed in 17 seconds, and build completed separately in 6 seconds. Our sandbox used lab-node:22 with 3 CPUs and 8 GB RAM. The harness records command success, exit code, elapsed time, timeout status, and a captured log tail. Parsed test counts are stored separately from command outcomes.
The test command exited 1 after 44 seconds without timing out. The attached stored parser summaries record 6 passed, 0 failed, 0 skipped, and 6 total. Those fields do not establish exhaustive unique test coverage. No new execution or complete-log reconciliation was performed for this correction.
The raw log identifies the failure in test/ui.test.tsx, in the Picker case named "moves with the arrows and keeps only the answer on screen after enter". At line 166, the assertion expected the final frame to contain "Vercel AI Gateway key" and received a newline. This establishes the failed assertion, but not its underlying cause or whether credential selection fails in an actual session. We have not substituted package-footer counts for the attached parser summaries or combined them into a new total.
The raw log marks test/piLifecycle.live.test.ts as skipped. The Pi extension guide explains that the normal suite skips this live repair test; opting in uses configured credentials and makes paid API calls. Skipped means untested here, not failed. Our run did not verify that live repair path, a deployment into a working agent session, model accuracy, or runtime latency.
The audit field is null. Advisory results and scan coverage are unknown. The repository scan recorded no CI workflow files and no Dockerfile, which describes repository shape rather than operational reliability.
MrKeyoor's testing methodology explains the distinction between command outcomes, parsed counts, and sandbox limits. It supplies provenance context; its current text does not prove that this older run used today's procedure unchanged.
What Abide checks
Abide compiles repository instructions into .abide/rubric.json and checks edits or completed activity diffs against that rubric. The README at the tested commit documents integrations for Claude Code, Codex, OpenCode, and Pi. These are documented capabilities, not four integrations independently exercised by our run.
A score of 0.8 or above requests repair. Scores from 0.5 to 0.8 produce a note, while lower scores remain quiet. These thresholds control how a judgment is delivered; they do not guarantee that the judgment is correct. The rubric has no built-in policy rules, so useful instruction files and review of their compiled questions are part of adoption.
Jev receives the rule and diff rather than the conversation. Choose initial rules whose violations a reviewer can recognize from the change itself. Treat instructions that depend on unseen repository context with more caution.
What happens when a check cannot run?
The tested README says every hook path exits 0. Missing credentials or network access allow the edit through unchecked, with the miss logged in .abide/events.jsonl. The Pi guide also states that checks happen after mutations and are not a pre-write security boundary.
Direct checks send changed lines to TypeSafe under your key. The README documents gateway routing and a configurable direct endpoint separately. Review the provider and retention arrangement that applies to your configuration before enabling checks on private code. Our sandbox did not validate those arrangements.
Does the replay prove agents repair violations?
The author's September 18, 2026 replay did not measure whether agents repaired flagged changes. It reconstructed edits from recorded Claude Code sessions rather than running those sessions again. Its reported reviewer was Claude reading each rule strictly, rather than an independent human audit.
The replay report describes 93 sessions, 1,256 judged edits, and 147 judged turns. Claude confirmed 10 of 39 edit flags and 11 of 15 turn flags, producing the report's precision figures of 26% and 73%. These are upstream replay results, not passing-test counts or measurements from our sandbox.
The sample used repository-specific rubrics, excluded shell-command changes, and did not establish accuracy on another repository. Its introduction and method also differ over the repositories' public or private status. The useful next step is to inspect your own rubric and feedback, rather than assume the reported precision transfers to your rules.
When should you choose Semgrep or ast-grep?
Semgrep is a static analysis tool that can run as a pre-commit check or in CI/CD workflows. Choose it when a requirement can be expressed in its analysis system and belongs at those workflow stages. Its README warns that Community Edition analysis is bounded by a function or file. It can replace an individual Abide check that it can express, while complementing Abide for other requirements.
ast-grep searches, lints, and rewrites code by matching syntax-tree structure. Choose it for a recognizable code pattern or structural rewrite. Its README establishes that role. It can replace model judgment for an expressible structural rule or run alongside Abide when other instructions still require judgment.
How to read the ratings
These are editorial ratings within each dimension: 1 means substantial obstacles, 2 means material limitations, 3 means usable with ongoing work or limited evidence, 4 means strong evidence with minor gaps, and 5 means unusually complete supporting evidence. They are not measured product performance or adoption scores.
Setup ease is 3: installation and build succeeded, but credentials, agent configuration, and rubric review remain. Documentation is 5 for its explicit treatment of setup, commands, data routing, failure behavior, and live-test limits. Community is 3 on limited evidence of public material and review activity; contributor breadth, response times, and adoption remain unverified. Maturity is 2 because our test command failed and live repair was not verified.
What to try first
Pin version 0.0.9, inspect the generated integration and rubric, and start with rules whose violations are easy to confirm. Review repair requests and logged unchecked edits before expanding use. Keep mandatory tests and static analysis in place. The next evidence to collect is whether your chosen agent completes a useful repair for your own rules.

