PR-AF turns one pull request into several review jobs
PR-AF does more than ask a model to summarize a diff. It maps the change, generates review dimensions, sends focused agents through those dimensions, extracts code evidence, tries to disprove each finding, and clusters related risks before producing comments. The result format includes severity, file, line, suggested change, evidence, and compound risk. That structure is useful because a reviewer can inspect why a warning exists instead of accepting a free-form model opinion.
The README presents benchmark leadership as the main sales case, with GLM and larger commercial models occupying different quality and cost positions. We did not rerun that benchmark, so it should be treated as the project's published evidence, not our measurement. The part a team can evaluate directly is the pipeline: reviewers have defined roles, findings face a falsifiability step, and output maps back to GitHub annotations.
The 40-second install is only the control-plane entry fee
Our commit 48ae7ee checkout installed 98 Python packages in 40 seconds and occupied 241 MB. The build passed in 4 seconds. The repository had 323 files, about 37,711 source lines, a Dockerfile, a Compose file, a tests directory, and 1 CI workflow. Pip-audit reported 0 known vulnerabilities in the installed Python environment. Those are sensible mechanics for a service that expects deployment rather than a one-off script.
A real review adds infrastructure and variable spend. PR-AF requires AgentField, an OpenRouter key, and a GitHub token that can read pull requests and post reviews. Railway supplies a control plane and PostgreSQL, while Docker Compose runs the stack locally. The API can be called asynchronously, and a public control plane needs its own API key. Operators also choose a model, wall-clock ceiling, cost ceiling, workspace root, and harness binary.
What happened when we ran it
Our sandbox reached the test suite after the 40-second install and 4-second build. Pytest ran for 14 seconds and reported 85 passed, 22 failed, and 6 warnings out of 107 tests. Every failure shown in the supplied log ended with the same class of message: async functions are not natively supported, followed by suggestions including anyio, pytest-asyncio, pytest-trio, and pytest-twisted.
The log tells us the async test environment was incomplete; it does not prove which plugin or configuration the maintainers intended. The practical result is still a failed suite with exit code 1. Passing 85 tests gives more signal than a checkout that never starts, but a CI review gate should not ask adopters to infer why 22 tests fail. Pinning the intended async test dependency and making the documented test command green would remove that uncertainty.
Pasted diffs have two open correctness failures
PR-AF accepts a pull-request URL, repository path, or diff text, but the open reports make those routes unequal. Issue 55 says a malformed hunk was silently read as an empty change and returned a successful review with zero findings. For downstream automation, that is worse than a loud parse error because a clean-looking result can pass a gate. The report asks for a failed execution that names the bad hunk.
Issue 56 describes a valid diff-only review where the pipeline identified a data race, then searched for the changed file inside PR-AF's own package directory. The verifier discarded the real finding and returned a location error after 20 invocations and 27 minutes. These are user reports, not results from our sandbox, but both concern advertised input behavior. Until fixed, provide a materialized repository workspace and reject empty results when the input contained changes.
Cost limits need a bill-side backstop
The configuration includes a per-run dollar ceiling and a 3,600-second duration default. Issue 57 reports that a review made 20 real model invocations over 27 minutes while total cost and every breakdown bucket remained zero. If accounting misses a provider path, the application-level dollar limit cannot be the only spend control. Set a provider-side budget, alerts, and concurrency limits before labeling many pull requests automatically.
Workspace use deserves similar attention. Issue 69 says concurrent agents still share one mutable checkout, where context files and agent output can race. The proposal is open rather than shipped. It discusses copy-on-write forks, cleanup, and isolation for commands that modify files. For current deployments, size the workspace volume, monitor accumulated clones, and avoid assuming that every reviewer can safely run destructive checks in parallel.
August maintenance is active, with no formal release
GitHub recorded the last push on August 21, 2026. The repository had 578 stars and 4 open issues and pull requests when fetched, with all 4 returned items being issues. There was no latest GitHub release. That combination shows active source work and a small, specific problem queue, while leaving adopters without a tagged release boundary to pin through GitHub Releases. Use a commit or image digest deliberately.
PR-AF is most convincing as a second review layer after formatting, types, tests, and security scanners. The 241 MB install is manageable, and its evidence schema is better than a generic model comment. The failed async tests, diff-only failures, and zero-cost report keep it out of unattended merge authority. Let it find things humans may miss, then require a human to judge the evidence.