A code reviewer that designs its own review
PR-AF does not send a pull-request diff through one fixed checklist. It first classifies the change, maps files and dependencies, and asks a planner to create review dimensions specific to that pull request. A payment endpoint might get reviewers for input validation and transaction consistency, while a logging refactor gets checks for behavioral preservation and missed call sites. Specialized agents then work in parallel, and later phases cross-reference, challenge, verify, and combine their findings.
The architecture makes sense for deep review. Static prompts waste attention on irrelevant categories, and one model pass can miss interactions across files. PR-AF tries to ground each claim with code context, reject findings that have an existing mitigation, and identify compound risks that only appear when two changes interact. Final findings include severity, location, explanation, suggestion, and evidence, and the service can post them as inline GitHub comments.
This is not intended as an editor companion. The README recommends using a quick local assistant during development and PR-AF as a final gate. Its stated 35-to-50-minute runtime makes that division necessary. A team should reserve it for consequential changes, scheduled review, or label-triggered audits rather than every small revision.
Setup comes in several similar-looking forms
Existing AgentField users get the cleanest path: install the repository, run the node, and call pr-af.review with a pull-request URL. New users must install the AgentField control plane first. Railway packages PR-AF, the control plane, and PostgreSQL. The root Docker Compose stack starts a local control plane and reviewer. GitHub Actions can launch that stack when a pr-af label is added.
Every path needs model and source-control credentials. OpenRouter is required by the documented default, while the GitHub token needs repository scope for reading code and posting reviews. Public control-plane calls need their own API key. Operators can cap duration, nominal model cost, concurrent reviewers, review depth, and coverage iterations. The default wall-clock ceiling is one hour, which fits the expected runtime but is a substantial CI allocation.
The implementation transition complicates this picture. Installing the bare repository through AgentField redirects to the maintained Go package on port 8007. Cloning and installing the local path deliberately retains the Python node, and the root Compose stack also runs Python. A separate Compose add-on can run Go beside it. The documentation explains the split, but teams can easily compare an issue, config default, or benchmark against a different implementation from the one they deployed.
The benchmark is useful evidence with boundaries
The repository includes a reproducible package for Martian Code-Review-Bench. PR-AF reviewed 38 runnable pull requests from five projects with one open model, without access to the human golden comments. The project reports 0.706 golden recall, second among 42 compared configurations overall and described as first among open-source reviewers. Its adjusted scoring reports 0.82 F1 and 595 valid findings, including non-golden bugs and valid nits.
Those numbers are more inspectable than a marketing chart. Inputs, outputs, scoreboards, per-pull-request judge decisions, and scripts are checked into the repository. The evaluation still has important limits. Twelve of 50 benchmark cases were deferred because their original pull requests could not be recovered. Comment matching and classification of extra findings use a language-model judge. The adjusted view was assembled by the PR-AF project, and benchmark leaderboards change. Treat the result as strong project-run evidence that merits reproduction, not a guarantee for a private codebase.
The README also claims a large cost advantage over closed tools, but the benchmark run used uncapped cost for quality. Measure spend on your own pull requests before relying on that claim. Model choice, diff size, review depth, parallel dimensions, and retries all change the bill.
Open defects affect the meaning of a clean result
The most troubling issues are not cosmetic. Issue 55 passes malformed diff_text with inconsistent hunk lengths. The Go reviewer swallows the parse failure, describes an empty pull request, and succeeds with no findings. Downstream automation can mistake that for a completed clean review. Issue 56 shows a valid diff-only review where the first phase noticed a data race, but the verifier searched PR-AF’s own package directory, decided the referenced file did not exist, and removed the real finding.
Cost enforcement also needs proof. Issue 57 records a 27-minute Go run with 20 model invocations and visible OpenRouter activity, yet every reported cost field remained zero. If that path cannot extract provider costs, the configured dollar ceiling cannot stop overspend. Use provider-side limits and alerts until local accounting is verified.
Parallelism has operational consequences. A 60,000-line lockfile-heavy pull request triggered enough harness fan-out to drive a Railway deployment toward its roughly 24 GB limit and crash. The August 10 fix added one shared agent budget, ignored generated files in expensive phases, and capped evidence work. A follow-up remains open because reviewers still share one mutable checkout, can race on context files, cannot safely run destructive tests, and leave cloned workspaces behind.
Active, young, and missing a crucial file
PR-AF was created in March 2026 and last pushed on August 10. Recent commits fixed large-PR retrieval, model-provider fallback, review fan-out, webhook controls, and the Go packaging switch. GitHub listed four open issues and two open pull requests. That is healthy activity, and the detailed incident reports show unusually candid engineering work.
There are no tagged GitHub releases, and both implementations identify themselves as version 0.1.0. More seriously, the README badge and Python package metadata declare Apache 2.0, but the repository has no LICENSE file and GitHub reports no detected license. Metadata is not a substitute for the actual license text and grant. Legal teams have a sound reason to stop there.
PR-AF is worth watching and testing because its adaptive review plan, adversarial filtering, and public results target the right problems. Today, it belongs beside human review as an experimental deep-audit service, not above it as an unquestioned gate. Require explicit parse failures, verify cost limits, isolate workspaces, and resolve licensing before expanding beyond a controlled pilot.