What our sandbox established
Our run on October 8, 2026 used commit d3bea6b in our sandbox. The full recorded commit is d3bea6b5793b5f3d59a75ebe69a58efa88383145. Installation succeeded, the build step returned zero, and the test command failed. This is a correction from the existing stored run, not a new execution.
Our harness records each command's success flag, exit code, elapsed wall-clock time, timeout status, and available log tail. It also stores parser summaries of test output. In this record, installation took 34 seconds, the build step took 5 seconds, and testing took 19 seconds. The empty build log tail and zero exit code do not demonstrate that a target image, deployment, or autonomous workload worked.
The MrKeyoor testing methodology explains this kind of provenance and its limits. Its supplied October 31 snapshot is background documentation; it does not establish that the October 8 run used that later procedure unchanged.
Docker blocked the patch tests
The raw tail lists four failures in tests/test_patch.py and ten in tests/test_patch_grade.py, plus an error in tests/test_patch_grade_e2e.py. Each reports FileNotFoundError for the docker executable. The failure therefore exposes an unmet execution prerequisite in our sandbox. It does not demonstrate whether those checks would pass with Docker available.
The attached stored parser summaries contain 411 passed, 14 failed, 5 skipped, 1 error, and a total field of 426. These are stored parser summaries, not verified exhaustive counts of unique tests. Their fields do not reconcile, and the repeated summary lines in the tail must not be counted again. Skipped tests remain skipped.
The raw pytest footer reports 12.04 seconds, while the harness records 19 seconds for the test command. These describe different recorded timings; neither replaces the other.
Pip-audit recorded zero advisories. Scan coverage is unknown: the record has a packages field of zero and no scannedPackages value establishing what was checked. This is not evidence that all 37 installed dependency packages were scanned or that the repository is secure.
Choose between static review and execution verification
The pinned README separates two useful entry points. Interactive threat modeling, scanning, and triage read and write files. Patching static findings also reads and writes files. The README requires reviewing and approving tool use in Claude Code and warns that static candidates on non-canary targets can contain more false positives because nothing is built or run.
The autonomous path executes target code. It requires sandbox setup and refuses agent-spawning operations outside gVisor unless explicitly overridden. Its reference detector uses Docker builds and ASAN for C/C++ memory errors. Moving to another stack means defining the finding signal, the proof-of-concept artifact, and the target build and execution process.
The verification boundary is worth studying: the README describes a separate grader reproducing the crash in a fresh container, with only the proof of concept crossing from the finding agent. That is documented design, not a result we demonstrated. Our run did not establish successful crash reproduction or the effectiveness of that boundary.
Start by defining the target's trust boundaries and reviewing the interactive artifacts. Before attempting autonomous execution, provide Docker, complete the documented sandbox setup, configure Claude access, and validate one target. Keep responsibility for severity, disclosure, and patch acceptance with a human owner. Anthropic says autonomous triage and patching remain open issues and verified patches are not always upstreamable.
Which alternatives answer a different need?
Semgrep supports static checks in an IDE, as a pre-commit check, and in CI/CD workflows. Choose it when the immediate requirement is integrating code checks into developer workflows rather than operating this autonomous reference pipeline. Its README warns that Community Edition analyzes within a single function or file and can miss security findings. We did not compare detection results.
The github/codeql repository contains the standard CodeQL libraries and queries that power GitHub Advanced Security. Consider it for a query-based analysis workflow, while checking the separate CLI license before analyzing closed-source code. The repository itself is not the CLI or engine, and our evidence does not establish comparative coverage.
OSV-Scanner finds existing vulnerabilities affecting project dependencies. It is a complement to investigating previously unknown source flaws: dependency advisory matching answers a different question. Choose it when known vulnerable dependencies are the immediate concern.
How to read the ratings
These are editorial assessments, not benchmark results. The shared scale is 1 for a major obstacle, 2 for substantial operator work or limitations, 3 for an adequate documented baseline, 4 for strong support with limited gaps, and 5 for comprehensive support within the dimension assessed.
Setup ease is 2 because Python installation succeeded but Docker blocked tests and autonomous execution needs additional infrastructure. Documentation is 5 for the breadth of the supplied README's workflow, sandbox, customization, and patching explanations; that score does not certify every linked instruction. Community is 2 because the project explicitly declines maintenance and contributions, regardless of attention. Maturity is 2 because it is a reference implementation with acknowledged gaps in autonomous triage and patching.

