mrkeyoor.comAsk

indada/repopilottested 05 Oct 2026commit 6a5597d

repopilot review

RepoPilot merits a bounded trial for maintainers who can supply trusted tests, explicit task selection and an isolated worker.

  • Install 56 s
  • Build 14 s
180/180tests passed
  • 21 packages
Automationevaluationupdated 26 Oct 2026
+0stars / 7d

RepoPilot is an independent, self-hosted controller built with the OpenAI Codex SDK. Its documented workflow turns selected GitHub issues and same-repository pull requests into test evidence, repair proposals and optional draft PRs. A separate Docker runner checks proposed changes; maintainers retain the merge decision.

Verdict

RepoPilot merits a bounded trial for maintainers who can supply trusted tests, explicit task selection and an isolated worker. Our October 5, 2026 run at commit 6a5597d completed dependency installation, build and the project test command successfully. That supports trying the source checkout, but does not validate live Codex calls, Docker repairs or GitHub publication. Keep merge decisions with maintainers and treat the worker as an environment for untrusted code.

We ran it

Lab card: what happened when we ran repopilot
Lab card from our sandbox run
Screenshot of repopilot (github.com/indada/repopilot)
github.com/indada/repopilot
Install✓ · 56s21 packages · 400 MB
Build✓ · 14s
Tests✓ · 226s180 passed · 0 failed of 180 (node:test)
Known vulns00 critical · 0 high · 0 moderate · 0 low (npm audit)
Repo138 files~7,504 lines of source · 0.6 MB · 2 CI workflows · tests dir

Answers from our run

Does repopilot build from source?

Dependencies installed in 56 seconds (21 packages), and the build succeeded in 14 seconds. We cloned commit 6a5597d into a clean Debian container with 3 CPUs and no project-specific setup.

Do repopilot's tests pass?

Yes. The project's own test command finished in 226 seconds and reported 180 passed, 0 failed (node:test).

Does repopilot have known vulnerabilities in its dependencies?

npm audit found none in the dependency tree at the time of our run.

Who should not use repopilot?

Repositories outside the documented public, text-only scope or above its 10,000-file / 16 MiB limits.

What are the alternatives to repopilot?

Aider, OpenHands. RepoPilot merits a bounded trial for maintainers who can supply trusted tests, explicit task selection and an isolated worker.

Setup3/5The dated installation, build and test commands succeeded; trusted images, Docker execution and credentialed operation still require operator setup and validation.
Docs4/5The supplied README, release, architecture and security extracts explain setup and verification limits in detail. Current v1.8.0 guidance must be separated from the measured v1.5.0 checkout.
Community1/5Insufficient evidence of adoption or support responsiveness. This is a confidence limitation, not a finding that the community is inactive.
Maturity2/5Portable v1.5.0 packages and successful local checks support evaluation, but the project calls itself a developer preview and the run excludes live model, Docker and publication workflows.

Who it’s for

Maintainers of public text repositories who want regression evidence and reviewable repair proposals.
Small teams prepared to configure trusted test commands, repository policy and execution limits.
Developer-tooling engineers evaluating structured Node, Vitest, pytest, Go or compatible JUnit XML evidence.
Codex SDK developers studying controller-owned permissions and recoverable task state.

Who it’s NOT for

Repositories outside the documented public, text-only scope or above its 10,000-file / 16 MiB limits.
Workflows requiring fork PR execution or browser E2E tests.
Operators unable to provide a dedicated development machine or VM without production credentials.
Teams requiring offline model inference: Codex calls send selected repository text and task context to OpenAI services.
Anyone expecting automatic merging or deployment.

Setup reality

In our recorded run on October 5, 2026, commit 6a5597d6c8e2bc4d61079bd7201f59ce7f86c9ce completed dependency installation in 55.674 seconds, the TypeScript build in 13.909 seconds and the test command in 226.103 seconds. The retained test output reported 180 passed, 0 failed and 0 skipped. These are recorded command results, without complete-log reconciliation of exhaustive unique coverage. They do not establish that a CLI binary was installed or that a live repair workflow succeeded.

The stored dependency summary records count 21 and 400 MB, but the installation tail says 12 packages were added. We cannot reconcile that discrepancy from the retained record. The recorded npm audit advisory total is zero; audit coverage is unknown, and zero findings do not establish safety.

The supplied README recommends Node.js 22, npm and Git for source setup, plus Docker with Linux containers for test and agent execution. OPENAI_API_KEY enables model calls. Publishing requires controller-held GitHub credentials with repository Contents and Pull requests write permissions. Portable v1.5.0 archives bundle Node.js and application dependencies, while Git and Docker remain external prerequisites. Provision trusted runner images and dependencies beforehand.

Passing project tests leaves the live repair workflow untested

RepoPilot's appeal is its documented separation of proposed changes from permission to publish them. Codex proposes tests and repairs; the controller requires runner evidence before publishing a separate branch and draft PR. Maintainers decide whether to merge.

For a public repository with reliable tests, that offers a concrete workflow to evaluate. Our recorded checks establish a narrower result: dependency installation, compilation and the project's test command completed successfully. They do not establish safe autonomous code delivery.

What our recorded run establishes

Our run began on October 5, 2026 at 05:51:52.225 UTC and checked commit 6a5597d6c8e2bc4d61079bd7201f59ce7f86c9ce. The build output identified repopilot-agent@1.5.0. The sandbox record specifies 3 CPUs, 8 GB of memory and image lab-node:22. That image label does not establish the actual Node runtime version.

Dependency installation exited successfully after 55.674 seconds, rounded to 56 seconds in the summary. The TypeScript build exited successfully after 13.909 seconds, rounded to 14. The test command exited successfully after 226.103 seconds, rounded to 226. Its retained output reported 180 passed, 0 failed, 0 cancelled and 0 skipped. We have not reconciled complete logs to establish exhaustive unique test coverage.

The stored dependency metrics record count 21 and 400 MB. The raw installation tail instead says 12 packages were added. Neither figure should be presented as a reconciled installed-package count. The npm audit record contains zero advisories across its severity fields. Coverage is unknown; this is not evidence that every dependency or the product is secure.

The checkout scan recorded 138 files, approximately 7,504 source lines, 0.6 MB, two CI workflow files and a tests directory. It did not detect a default Dockerfile. The README separately provides Dockerfile.agent for the agent image.

The pinned README says the project tests use mocked APIs, agents and runners alongside synthetic local Git/Node fixtures. They do not use model credits, launch Docker or write GitHub content. Our run therefore does not verify a live model call, target-repository workload, Docker repair, installed CLI binary or published PR.

What the verification design promises

According to the README, RepoPilot pins revisions and trusted base-branch policy, freezes generated tests before repair and checks case identities across executions. A base-pass/head-fail result can identify a regression. A base-fail/head-pass result qualifies as new behavior only with supporting requirement evidence. Ambiguous failures, missing cases and execution errors require review.

The architecture document adds an important limit: repeated failure identities and fingerprints detect changing failures, but do not provide statistical proof that a test is stable. Generated tests also cannot establish that a proposed behavior matches the maintainer's intent. Human review remains necessary.

Trusted .repopilot/policy.json rules can check literal text and direct or qualified calls. Nested AGENTS.md files inform semantic review. The documented AST rules do not resolve aliases or perform whole-program analysis, and prose conflicts require human interpretation.

Keep the measured checkout separate from current documentation

The recorded build identifies v1.5.0. Its release notes describe optional Planner, Tester, Developer and Reviewer Codex roles, shared call/token accounting, durable handoffs, post-merge tracking and stored-evidence evaluation. Portable archives include Node.js and dependencies, but require external Git and Docker. The packages are not code-signed or notarized.

The supplied current README describes v1.8.0. It includes optional bounded parallel execution of independent task-graph nodes, a persistent service with signed Webhooks and polling, and a local read-only console. Those are documented current capabilities, not results from our v1.5.0 checkout run. A blanket claim that RepoPilot has no Webhooks or dashboard would therefore be misleading. The current documentation still excludes fork execution, browser E2E, automatic dependency installation, hosted administration and automatic merging.

The worker needs its own boundary

The security guide recommends a dedicated development machine or VM without production credentials. Test containers have no network by default; configured dependency services use a disposable internal Docker network. Containers share the worker kernel and do not provide a complete boundary against hostile code.

The agent receives an OpenAI key and repository text, while GitHub credentials remain with the controller. Self-hosting does not make inference offline. Local reports and snapshots can contain code and logs, and the preview does not claim comprehensive secret redaction. Test code shares a container with its runtime and reporter, so structured evidence is not tamper-proof attestation against malicious code.

How to decide

Start with a local check on a public text repository, using a trusted runner image and pre-provisioned dependencies. Keep publishing disabled while reviewing the resulting evidence. Model calls, Docker execution and GitHub publication each need separate validation in your environment.

Aider suits direct terminal pair programming. OpenHands Agent Canvas suits operators who want a control center spanning local, remote and cloud agent backends. RepoPilot's Codex SDK dependency is part of its implementation; an SDK alone is not a substitute for operating this controller.

Rating rubric and evidence limits

These ratings describe readiness for the supervised public-repository workflow above. A score of 1 means substantial uncertainty or missing evidence; 2 means a limited basis for evaluation; 3 means a usable starting point with material operator work; 4 means detailed support for this workflow with remaining gaps; 5 requires comprehensive evidence for the dimension. Unknowns lower confidence rather than establish product defects. Ratings are editorial judgments, not benchmarks or security certification.

Setup earns 3/5 because local commands succeeded but Docker and credentialed operation remain untested. Documentation earns 4/5 for specific setup, verification, recovery and security guidance, with version differences requiring care. Community earns 1/5 because the supplied evidence does not establish adoption or support responsiveness. Maturity earns 2/5 because packaged releases and local checks provide a basis for evaluation, while the project remains a developer preview and our run excludes live operation.

Alternatives

ProjectWhat it isPick it when
Aider gh↗A terminal tool for pair programming with language models.Choose Aider when you want to request changes directly in a coding conversation within your existing codebase.
OpenHands gh↗Agent Canvas is a self-hosted control center for coding agents and automations.Choose OpenHands Agent Canvas when you need a frontend that can switch between local, remote and cloud agent backends.

What people are saying

  1. [velocity-scout] indada/repopilot

Sources

  1. RepoPilot README
  2. RepoPilot v1.5.0 release
  3. RepoPilot security model
  4. RepoPilot architecture

More automation reviews