mrkeyoor.com_
Wed 11 Nov 13:16 UTC
Dev Toolsevaluationupdated 24 Oct 2026

pr-af review

PR-AF is a self-hosted AI pull-request reviewer built on AgentField. Its README describes focused review agents, code-evidence checks and attempts to invalidate findings before posting GitHub comments. The maintained installation is now the Go node; our recorded sandbox run covered the Python checkout, not that Go deployment.

+1stars / 7d
Verdict

PR-AF merits a supervised trial if its agent-based review approach fits your workflow. Our October 9, 2026 Python run at commit 421fbd2 completed its install, build and test steps successfully, with pytest reporting 126 passed. That result does not validate the maintained Go installation or establish review quality, runtime or cost. Test your intended deployment and input route before assigning it approval authority, and reconcile reported spending with provider records.

We ran it

Lab card: what happened when we ran pr-afScreenshot of pr-af (www.agentfield.ai)
Install✓ · 15s111 packages · 359 MB
Build✓ · 1s
Tests✓ · 15s126 passed · 0 failed of 126 (pytest)
Known vulns0(pip-audit)
Repo328 files~38,746 lines of source · 10.9 MB · 1 CI workflows · Dockerfile · tests dir

Answers from our run

Does pr-af build from source?

Dependencies installed in 15 seconds (111 packages), and the build succeeded in 1 seconds. We cloned commit 421fbd2 into a clean Debian container with 3 CPUs and no project-specific setup.

Do pr-af's tests pass?

Yes. The project's own test command finished in 15 seconds and reported 126 passed, 0 failed (pytest).

Does pr-af have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use pr-af?

Teams seeking evidence here that the maintained Go node has passed deployment or end-to-end review checks. Our recorded run covered Python setup, build and pytest steps.

What are the alternatives to pr-af?

PR-Agent, reviewdog. PR-AF merits a supervised trial if its agent-based review approach fits your workflow.

Setup3/5The October 9, 2026 Python install, build and test steps succeeded, but service deployment and the maintained Go installation were not verified. AgentField and provider credentials add operational work.
Docs4/5The supplied README explains Go versus Python installation, CLI and HTTP entry points, Railway, Compose, credentials and configuration defaults. Documentation detail does not establish that each route was tested.
Community2/5The supplied issue extracts show concrete problem reports, but do not establish adoption, response times or resolution rates. Historical star and queue counts do not support a current community rating.
Maturity2/5The Python run completed successfully, but the maintained Go deployment and actual review workloads remain unverified here. Supplied open Go-path reports warrant validation of input handling and provider accounting.

Who it’s for

Teams evaluating an additional review layer after their existing automated checks.
Developers willing to supply model credentials, operate AgentField and assess findings themselves.
Operators who can validate the chosen implementation, input route and provider accounting before automating reviews.

Who it’s NOT for

Teams seeking evidence here that the maintained Go node has passed deployment or end-to-end review checks. Our recorded run covered Python setup, build and pytest steps.
Users who need independently measured review accuracy, latency or model costs. We did not measure those workloads.
Teams expecting application-reported costs or an empty review to be sufficient approval evidence. Supplied Go issue reports describe failures in those paths.

Setup reality

In our recorded run on October 9, 2026, at commit 421fbd23bf1c5a2c3916d7046c97b5273958e1f4, the Python dependency-install step completed in 15.464 seconds, the build step in 0.656 seconds and the test step in 15.485 seconds. The stored dependency summary records 111 packages and 359 MB. Pytest reported 126 passed; the parser summary records zero failures, skips and errors. These are recorded command results, not independently reconciled exhaustive test coverage.

The install tail identifies Python 3.12.12. The build tail is empty, so its successful exit does not establish which artifacts were produced. The record does not establish an installed CLI check, a running control plane, a Go build or an actual model-backed pull-request review.

The README says installing the bare repository URL through AgentField selects the maintained Go implementation. Python remains available through a local-path installation, python -m pr_af.app or the root Docker Compose stack. Deployment requires AgentField, OPENROUTER_API_KEY and GH_TOKEN; the Railway option also deploys PostgreSQL, and its public endpoint requires an API key.

The recorded checkout has a Dockerfile, Compose file, tests directory and one CI workflow. Pip-audit recorded an advisory total of zero, but audit coverage and the number of scanned packages are unknown. This is not proof of safety.

A passing Python run leaves a Go deployment question

PR-AF's maintained installation and our measured checkout need separate evaluation. The README says a bare repository URL installed through AgentField selects the Go node. Our October 9, 2026 sandbox record covers the Python checkout at commit 421fbd23bf1c5a2c3916d7046c97b5273958e1f4. Its successful commands are useful setup evidence, but they do not establish that the maintained deployment works end to end.

PR-AF is intended for teams that want an additional AI review layer and can operate the service themselves. The README describes generating review dimensions, assigning focused agents, grounding findings in repository evidence and challenging them before producing GitHub comments. Those are documented design claims. We did not measure their effectiveness on pull requests.

What our recorded run establishes

The stored run used a sandbox configured with 3 CPUs and 8 GB of memory. Its install tail identifies Python 3.12.12. The dependency-install step exited successfully after 15.464 seconds, with 111 packages and 359 MB recorded in the dependency summary. The build step exited successfully after 0.656 seconds; its empty output tail does not identify a built artifact.

The test step exited successfully after 15.485 seconds. Its pytest tail says "126 passed in 13.35s"; the parser summary records 126 passed, zero failed, zero skipped and zero errors. The step duration and pytest's own duration measure different boundaries. We have not reconciled a complete log to establish exhaustive unique test coverage.

The checkout summary records 328 files, 38,746 source lines and 10.9 MB, plus a Dockerfile, Compose file, tests directory and one CI workflow. Pip-audit recorded zero advisories. Its coverage is unknown, so neither the advisory total nor the dependency count establishes that every dependency was checked.

There is no recorded installed-binary check, control-plane deployment, Go test run, model-backed review, GitHub comment delivery or measured review workload. These results cannot establish review accuracy, production reliability, cost or latency.

Choose the implementation before following setup instructions

According to the README, installing the repository URL through AgentField follows the manifest to the maintained Go package. Python remains available through a local-path install, python -m pr_af.app and the root Docker Compose stack. Select the route deliberately rather than treating our Python results as validation of both implementations.

The documented service needs AgentField, OPENROUTER_API_KEY and GH_TOKEN. Railway deploys the reviewer, control plane and PostgreSQL; its public execution endpoint requires X-API-Key. Operators also need to choose the provider, model, workspace location and execution limits.

The README lists default limits of $2.00 per run and 3,600 seconds. These are configuration defaults, not measured spending or guarantees. It describes reviews as typically taking 35-50 minutes, which we did not independently time. Its benchmark leadership and cost comparisons are also project claims, not findings from our sandbox.

Validate empty results and provider accounting

The supplied extract for issue 55 is open and reports a malformed diff returning a successful empty review. The report concerns pr-af-go v0.1.0 on macOS, not our Python sandbox run. It supports checking that invalid input fails clearly; it does not establish that every current installation has the same behavior.

Issue 56, also supplied as open, reports a valid diff-only review without repo_path. The reporter says verification searched the agent's package directory, dropped a data-race finding and returned a location-related finding after 27 minutes and 20 invocations. This is a reported Go-path failure, not a reproduced result from our run. Validate diff-only behavior separately from repository-backed reviews.

Issue 57 describes the same Go review using the OpenCode provider: 20 invocations over 27 minutes while the reported cost remained zero. That report does not establish failure across all providers. It does justify reconciling application accounting with provider records before relying on an application-level budget ceiling.

For a trial, use a pull request with changes you understand. Check that the service reviewed the intended files, inspect the evidence behind its findings and compare reported spending with provider records. Keep existing automated checks and human approval while evaluating those results.

Alternatives serve different review jobs

PR-Agent's README documents an open-source AI assistant with review, description and improvement commands, CLI use, GitHub Actions and several Git-provider integrations. It identifies the project as community maintained and distinct from Qodo's current offering. Consider it when those commands and integration routes fit your workflow; this review does not establish comparative accuracy or speed.

reviewdog takes output from lint tools, filters findings against a diff and posts comments to code-hosting services. Choose it when you need existing diagnostics delivered into review. It does not serve the same purpose as PR-AF's documented agent-based analysis.

How to read the scores

These scores assess the supplied evidence for adoption: 1 means major unresolved obstacles, 2 means limited evidence with consequential unknowns, 3 means a practical evaluation path with operational work, 4 means detailed support for the assessed area, and 5 would require broad corroboration beyond this record. They are editorial judgments, not benchmark results. Unknown adoption and untested compatibility do not earn a high score.

The next useful check is a supervised review through the implementation and provider you intend to deploy. Our Python command results support proceeding to that evaluation; they do not replace it.

Alternatives

ProjectWhat it isPick it when
PR-Agent gh↗A community-maintained, open-source AI review agent with review, description and improvement commands.Choose it when you want a PR assistant documented for CLI or GitHub Actions use, with several Git-provider integrations. Its README distinguishes this legacy community project from Qodo's current offering.
reviewdogA tool that filters linter output against a diff and posts review comments.Choose it when you already have lint or analysis tools and need their diagnostics delivered into code review. It serves a different role from a model-led reviewer.

What people are saying

  1. [github-trending] Agent-Field/pr-af

Sources

  1. PR-AF README
  2. Malformed diff succeeds empty
  3. Diff-only verification drops a finding
  4. Cost accounting reports zero
  5. Shared workspace isolation proposal

More dev tools reviews

folly · universal-android-debloater-next-generation · OpenSandbox · You-Dont-Know-JS · evilginx2 · google-colab-cli · the whole board →