mrkeyoor.comAsk

thruwire/foremantested 04 Oct 2026commit e5d1aa4

foreman review

Use Foreman for an inspected trial of single-worker supervision, starting with the documented deterministic demo and then a bounded repository job.

  • Install 26 s
  • Build 4 s
259/260passed, run exited non-zero
  • 63 packages
Automationevaluationupdated 31 Oct 2026
+3stars / 7d

Try Foreman for bounded repository jobs when you want a separate supervisor around an existing coding agent and can inspect its interventions. Foreman supervises one coding worker at a time. It assesses progress, completion, verification needs and human intervention; the worker still chooses tools and edits files. Our run on October 4, 2026 tested commit e5d1aa4 in our sandbox, not authenticated coding-agent supervision or assessment accuracy.

Verdict

Use Foreman for an inspected trial of single-worker supervision, starting with the documented deterministic demo and then a bounded repository job. Our October 4, 2026 run at e5d1aa4 established successful installation and build alongside a failing test command. It did not establish reliable autonomous completion, calibrated assessments or successful real-CLI supervision. Wait if those are prerequisites for your deployment.

We ran it

Lab card: what happened when we ran foreman
Lab card from our sandbox run
Screenshot of foreman (thruwire.ai)
thruwire.ai
Install✓ · 26s63 packages · 88 MB
Build✓ · 4s
Tests✗ · 47s259 passed · 1 failed of 260 (pytest)
Known vulns0(pip-audit)
Repo95 files~12,609 lines of source · 0.6 MB · 1 CI workflows · tests dir

Answers from our run

Does foreman build from source?

Dependencies installed in 26 seconds (63 packages), and the build succeeded in 4 seconds. We cloned commit e5d1aa4 into a clean Debian container with 3 CPUs and no project-specific setup.

Do foreman's tests pass?

No. The project's own test command finished in 47 seconds and reported 259 passed, 1 failed (pytest).

Does foreman have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use foreman?

Teams requiring calibrated assessment accuracy: the project explicitly says this remains unproven.

What are the alternatives to foreman?

Aider, LangGraph. Use Foreman for an inspected trial of single-worker supervision, starting with the documented deterministic demo and then a bounded repository job.

Setup3/5Usable with material prerequisites: our install and build succeeded, but real supervision requires provider credentials and a configured coding-agent CLI.
Docs5/5Unusually complete documentation of runtime, policy, evidence, worker backends and execution limits in the supplied extracts; this rating does not validate assessment accuracy.
Community3/5Limited assessment based on a published release and contribution guidance. Adoption, contributor breadth and support responsiveness are unknown; the rating does not measure popularity.
Maturity3/5Offline tests and explicit guardrails support evaluation, but our test command failed and documented limits include unproven accuracy, local execution and non-production durability.

Discussed on

  1. hnForeman: An agent supervisor and software factory foreman3 points

Who it’s for

Developers who already use a coding-agent CLI and want explicit supervision around its work.
Operators willing to review evidence, tune thresholds and inspect stopped runs.
Codex App Server users who need guidance delivered during an active turn.
Researchers evaluating supervision policies without treating model scores as proof of correctness.

Who it’s NOT for

Teams requiring calibrated assessment accuracy: the project explicitly says this remains unproven.
Operators requiring concurrent coding workers or production-grade durable execution.
Users expecting Foreman to isolate untrusted repositories or jobs.
Users requiring live steering through backends other than Codex App Server.
Release gates requiring a successful test command at the recorded commit: our sandbox test command exited 1.
Teams choosing on demonstrated real-workload reliability: our run did not measure that.

Setup reality

Our run on October 4, 2026 used commit e5d1aa4 in a Python 3.12 Debian Bookworm sandbox with 3 CPUs and 8 GB RAM. Installation succeeded in 26 seconds, adding 63 packages with an 88 MB footprint. The build command succeeded in 4 seconds; the test command exited 1 after 47 seconds.

The attached stored parser summaries report 259 passed, 1 failed and 260 total, with no skipped tests or errors recorded. These summaries do not establish exhaustive unique test coverage. No new execution or complete-log reconciliation was performed for this correction.

At the tested commit, real supervision required Python 3.11 or newer, a TypeSafe API key and a configured coding-agent CLI. The offline suite required no credentials, network, Codex process or external repository. Successful installation and build do not demonstrate a real supervised job.

Decide whether you need a supervisor

Foreman is worth a bounded trial if your coding agent already does the engineering work and you want another process to decide when it should continue, receive guidance, stop or seek verification. Keep a person available to inspect those decisions. The project says assessment accuracy is unproven and its semantic scores need calibration.

Foreman supervises one coding worker at a time. Its Python policy selects actions from model-informed proposals; the assessment model does not choose individual files or tools. This separation gives operators a place to configure intervention rules, but it does not establish that the model judges completion correctly.

What our sandbox established

Our run on October 4, 2026 tested commit e5d1aa4. Installation and build succeeded in our sandbox; the test command exited 1. The harness records command success, exit status, elapsed time, timeout status and parsed test counts separately. A successful command therefore describes that step, not an authenticated deployment or completed software job.

The stored parser summaries report 259 passed, 1 failed and 260 total. They record no skipped tests or errors. These are stored parser summaries, not verified exhaustive counts of unique tests. We have not rerun the project or reconciled a complete log for this correction.

The raw failure identifies tests/test_integration.py::test_noisy_events_are_coalesced. Its assertion expected 50 persisted output events and found 24. The excerpt uses a fake model and worker. It establishes that assertion failure in our sandbox, without establishing whether scheduling, implementation or the expectation caused it. The raw pytest footer reports 40.71 seconds, while the harness records 47 seconds for the test command; these describe different timing scopes.

Our run did not test authenticated Codex, OpenCode or Hermes supervision, live steering against a real CLI, assessment calibration or completion of an external repository job. The pinned README describes the suite as offline. Its deterministic demo is a useful next step, but the supplied record does not establish that we separately ran the demo.

The audit record contains zero reported advisories from pip-audit. Scan coverage is unknown because the record does not supply scannedPackages; its packages field is zero. This is not evidence that every installed package was checked or that Foreman is secure.

The MrKeyoor testing methodology explains the distinction between command outcomes and workload evidence. That page supplies provenance context; its current procedure does not establish which methodology version governed our October 4 run.

Choose the backend for the intervention you need

At the tested commit, Foreman supported Codex, OpenCode and Hermes workers, with Jev supplying assessments. Codex App Server kept an active turn addressable for steering or interruption. The Codex exec transport could not accept live steering, and the other backends fell back to stop and retry. If guidance during a running turn is your reason to adopt Foreman, backend choice matters before installation.

The supplied current README describes additional capabilities: an OpenAI Decisions API adapter, Deep Agents Code workers and attached supervision for supported sessions including Pi and Pi Durable. Those are current documentation claims, not capabilities exercised by our older run. The 0.4.1 release page also identifies a different release commit, so the release tag and our tested checkout should not be treated as identical.

Set permissions before autonomous work

Foreman does not isolate local execution. Workers retain local-environment permissions; Codex requests its workspace-write sandbox, while OpenCode runs with --auto subject to its permission configuration. Configured evidence commands also inherit local permissions and Foreman's environment, although Foreman invokes them without a shell and bounds their time and retained output.

Review the repository, worker permissions and trusted evidence commands before starting a real job. Use a task whose changes you can inspect and whose stopped state you can recover deliberately. Persistence supports inspection, but the README explicitly excludes production-grade durable execution. The supplied evidence does not independently verify the older page's issue-specific account of escalation recovery, so this review makes no claim about that issue's current status.

Alternatives answer different questions

Aider is a terminal pair programmer for starting a project or editing an existing codebase. Choose it when you want to work directly with a coding assistant rather than add Foreman's supervisory layer. This is a choice of workflow, not evidence that Aider implements Foreman's policy or is a documented Foreman backend.

LangGraph is a low-level framework for building long-running, stateful agents. Choose it when you intend to build the workflow and its state transitions yourself. Its documentation describes durable execution and human oversight, but adopting a framework entails implementing your own application; it is not a drop-in Foreman replacement.

Your existing coding-agent CLI is a complement to Foreman: it remains the worker. Neither alternative comparison establishes compatibility with Foreman or measures which approach produces better code.

How the ratings should be read

These editorial ratings use a 1 to 5 rubric: 1 means substantial documented obstacles, 2 means major prerequisites or gaps, 3 means usable with material limits, 4 means strong supporting evidence with limited gaps, and 5 means unusually complete evidence for that dimension. They are judgments about the supplied evidence, not benchmarks or adoption measurements.

Setup ease is 3 because installation and build succeeded, while real use still requires credentials and a configured worker. Documentation is 5 for its detailed treatment of runtime, evidence, policy, backends and execution limits; this does not rate the truth of model judgments. Community is 3 on the narrow evidence of a published release and contribution guidance; contributor breadth, response times and adoption remain unknown. Maturity is 3 because the project has offline tests and explicit guardrails, while our test command failed and the README excludes calibrated accuracy, isolation and production-grade durability.

Alternatives

ProjectWhat it isPick it when
Aider gh↗A terminal pair programmer for working directly on a new or existing codebase.Choose Aider when you want direct interaction with a coding assistant; its supplied documentation does not establish a Foreman-style supervisory layer or Foreman integration.
LangGraph gh↗A low-level framework for building long-running, stateful agent workflows.Choose LangGraph when you plan to implement your own workflow, persistence and human oversight. It is a framework alternative, not a drop-in replacement for Foreman.

What people are saying

  1. [velocity-scout] thruwire/foreman

Sources

  1. Foreman README
  2. Foreman 0.4.1 release
  3. Foreman runtime and event flow
  4. Issue 40: recovery after human escalation

More automation reviews