Our November 19, 2026 run installed Balanov/verdict in 23 seconds and passed all 14 tests. Use it if you are building a coding agent and want a local second opinion on command output. Skip it if a classifier's success label must authorize a deployment: the README describes Verdict as an advisory classifier, with deterministic checks and authoritative state verification as the source of truth.
Balanov/verdict reads command-line evidence and assigns one of four labels. The adoption question is whether that extra opinion helps your agent choose between retrying, fixing code and investigating an environment problem. Issue #1 gives that question concrete inputs: an HTML HTTP error, a Bun database TypeError and a Cargo summary reporting zero tests. The maintainer reproduced the reports, and two became training corrections. Those cases explain the classifier's behavior more clearly than a feature list alone.
What you can do with verdict
Balanov/verdict classifies a command, exit code, stdout and stderr as success, failure, blocked or unknown. Its output contract defines success as positive evidence of completion, failure as an operation reporting a concrete failure, blocked as an execution obstacle, and unknown as absent or inconclusive evidence. These distinctions give an agent more categories than a numeric process status alone, particularly when the output describes skipped work or an environment problem.
The README demonstrates both a Python API and a CLI. Its Python example passes pytest tests/, exit code zero and output saying no tests ran; the illustrated result is unknown, with advisory: true. The CLI accepts piped output and options for stdout, stderr and a weights file. These documented entry points let an integration preserve the command's identity alongside its output instead of classifying an isolated error sentence without context.
What people use verdict for
GitHub reporter alloevil tested Verdict on real curl, Bun and Cargo outputs, and maintainer Balanov used the counterexamples to expand training coverage in issue #1. The thread documents two distinct activities: evaluating labels against known command outcomes and turning disagreements into regression cases. It also records a retracted observation, which is useful because the corrected Cargo result changes how the issue should be read.
| Person | What they did | Outcome |
|---|---|---|
| GitHub reporter alloevil | Compared curl, Bun and Cargo outputs across a preserved installation and source revisions. | Reproduced an HTTP error labeled success and a caller-code error labeled blocked; retracted the earlier Cargo unknown result. Reporter’s corrected observations |
| Maintainer Balanov | Reproduced the reports, expanded synthetic training families and retained real traces as regression fixtures. | Reported that the HTML HTTP error and database TypeError classify as failure after merged PR #2. Maintainer’s resolution |
For an integration, the useful practice is to save the complete input together with an independently chosen expected label. The Cargo exchange shows why: a short summary left a discrepancy between the reporter's initial probability and the maintainer's reproduction. Repeating the inputs across installations resolved the observation without attributing it to a software change. A regression collection should preserve both the original disagreement and the corrected result rather than silently replacing expectations with the model's answer.
What happened when we ran it
Our November 19, 2026 source-checkout run at commit b30f536 installed in 23 seconds, built in 1 second and completed tests in 2 seconds. Pytest reported 14 passed, 0 failed of 14. The sandbox configuration recorded 3 CPUs, 8 GB RAM and image label ghcr.io/astral-sh/uv:python3.12-bookworm. The checkout contained 47 files, approximately 995 source lines and 2 MB of files, making the measured installation easy to distinguish from the earlier registry attempt.
| Recorded question | Answer | Verdict | Why |
|---|---|---|---|
| Did source installation finish? | Yes, in 23 seconds; 36 dependencies and 39 MB recorded. | Passed | The installation step completed successfully. |
| Did the build finish? | Yes, in 1 second. | Passed | The recorded build step succeeded. |
| Did the tests pass? | Yes, in 2 seconds; pytest reported 14 passed, 0 failed of 14. | Passed | All tests counted by that command passed. |
| What did the dependency audit report? | pip-audit reported 0 known vulnerabilities. | Audit completed | This is the recorded audit result. |
| What repository infrastructure appeared in the scan? | Two CI workflow files, a tests directory and no Dockerfile. | Infrastructure recorded | These are source-tree scan findings. |
The source-run results come from our sandbox measurements. A separate hands-on record says the quickstart-cli clone step took 1 second. That earlier run used commit b30f5362b3b4, a fresh unprivileged container with 2 CPUs and 4 GB RAM, no published ports and no secrets, according to its run record. Its configuration differs from the successful source-checkout harness above.
In that earlier run, uv tool install verdict-tiny failed in 0.7 seconds with exit code 1. The resolver said verdict-tiny was not found in the package registry. The installation finding recorded unsatisfiable requirements and stopped at installation, and teardown completed successfully. The README's documented installation path is a source clone followed by python -m pip install ., which is a different path from that registry command.
Where people get stuck: an error's wording changes the label
An HTML HTTP 500 response received success with confidence 0.734204 in Balanov's reproduction in issue #1. The thread identifies a training-coverage difference: the old corpus represented the curl failure in another form, while this counterexample used an HTML body. The maintainer added the curl_500_html family in merged PR #2 and reported that the example then classified as failure.
The Bun example crossed a different boundary. A read-only database TypeError received blocked with confidence 0.968427, although alloevil identified the underlying Bun 1.4.4 readonly-plus-create error as a caller-code bug. Balanov reported that the existing read_only family had absorbed the example into blocked. The added code_readonly_db family addressed that distinction, and the maintainer reported failure for this example after the same merged correction in issue #1.
| Symptom | What issue #1 established | Integration decision |
|---|---|---|
| HTML HTTP error receives success | Reproduced at confidence 0.734204; maintainer reports a merged training correction. | Use the HTTP response status as the endpoint's direct outcome check. |
| Database TypeError receives blocked | Reproduced at confidence 0.968427; maintainer reports a merged training correction. | Give caller-code failures and environment obstacles different recovery actions. |
| Cargo reports zero tests and receives success | Reproduced as success; the earlier unknown observation was retracted. | Include the intended test count or execution evidence in the acceptance condition. |
| Unreachable host receives failure | Maintainer describes this as an intentional training trade-off. | Assign retry behavior according to the command's expected outcome. |
The Cargo correction is especially important when reading the issue's history. Alloevil initially reported unknown at 0.531, then repeated the fields against a preserved verdict-tiny 0.1.0 installation, initial release source a5880d9 and main f7fc731. The corrected result was success 0.600991, with unknown 0.391798. The reporter retracted the earlier observation and said the inference source and weights were byte-identical across those versions in the corrected Cargo report.
Version 0.1.1 rejects incompatible model artifacts
Version 0.1.1 adds rejection of incompatible feature hashes, according to the release notes. The release page identifies commit b30f536, matching the short commit recorded by our source-run harness. This establishes a concrete version connection for the reviewed checkout rather than assuming that a current README and an arbitrary local installation describe the same release.
Balanov reports in issue #3 that merged PR #4 compares the artifact's declared feature hash with the expected hash before reading weights. A mismatch raises ValueError naming both values, and one constant now supplies the expected hash to both loader and trainer. The maintainer also reports identical predictions for both shipped artifacts. This check concerns whether an artifact matches the feature recipe used by the installed code.
Synthetic accuracy describes a particular evaluation
The README reports 93.2% accuracy and 93.2% macro-F1 for Verdict 0.1 int8 on 192 synthetic examples from 48 held-out CLI families. Training used 1,440 synthetic examples from 60 other families. Its published exit-code baseline scored 47.9% accuracy and 31.9% macro-F1. These are the project's benchmark figures, separate from our installation, build and test measurements at b30f536.
The same benchmark reports a 197 KB model and median inference of 0.54 ms. The README places that latency on Windows 11, AMD64 and Python 3.12, using 576 predictions including feature extraction and the Python API call. In issue #1, Balanov reports a later held-out score change from 0.9323 to 0.9375, with macro-F1 moving from 0.9322 to 0.9374 after the training corrections and real traces retained outside training as regression fixtures.
The update also records a deliberate category choice: the offline family now maps to failure rather than blocked because the maintainer does not treat an unreachable host as a policy refusal. That decision appears in issue #1's resolution, alongside the two corrected examples. An agent's recovery policy therefore needs a mapping from Verdict's categories to its own actions; the label names alone do not define whether to retry a network request or edit code.
Local inference uses bounded command evidence
The README describes inference as pure Python standard-library execution with local weights and no network calls or telemetry. The model uses hashed character and word features with multiclass logistic regression. Its parameters include 196,608 int8 weights, four biases and per-class scales. This is a local classifier for command evidence, with a documented runtime different from a hosted language-model service that receives each shell log.
The input contract bounds stdout and stderr to 12,288 characters per stream with explicit head/tail clipping. That gives an integration a concrete input budget and makes the original log a separate diagnostic artifact worth retaining. The README also describes a browser demo using the same quantized weights without uploading logs. Code and model weights use Apache-2.0, while the synthetic dataset uses CC0-1.0, according to the license section.
verdict vs pytest, Pexpect and exit-code checks
Use Verdict for advisory classification of varied output, pytest for explicit assertions, Pexpect for interactive patterns, and exit codes when they adequately express the required result. The pytest README documents plain assertions and fixtures; the Pexpect README describes spawning and controlling child applications while responding to expected output patterns. The selection depends on whether the desired outcome is already expressible as a test or known interaction.
| Approach | What it evaluates | Pick it when |
|---|---|---|
| Balanov/verdict | Command, exit code, stdout and stderr through a learned classifier. | You want an advisory label across varied command output. Verdict output contract |
| pytest | Tests using assertions and fixtures. | You can express the required behavior directly. pytest README |
| Pexpect | Child-process interaction and expected output patterns. | You need to drive an interactive CLI and react to prompts or text. Pexpect README |
| Exit-code check | The process's numeric return status. | That status adequately represents the required result. |
For a code change, an assertion can express the behavior required for acceptance. For an interactive installer, an expected prompt can determine the next input. Verdict addresses a different integration problem: interpreting varied command output through one classifier rather than writing a parser for each command. These approaches can sit in the same agent, with explicit outcomes deciding completion and the classifier helping organize the text the command produced.
Balanov/verdict is separate from similarly named services
Balanov/verdict is the CLI-output classifier; Verdict Reviews describes reviewing changed files in GitHub, while VERDICT at verdictreview.co.uk describes auditing agency proposals. Those descriptions identify different products and services. Use the full repository name when looking for this classifier's issues and release history. The existing openJev-verdict-2.0 review also covers a separate repository and model project.
FAQ
Issue #1 records confident mistakes on real CLI output and maintainer-reported corrections for two examples, so Balanov/verdict's intended role is advisory classification. The six questions below concern reproducing those results, interpreting the output categories and selecting compatible artifacts. They follow the software questions raised by the issue discussions rather than treating unrelated searches for the word verdict as questions about this Python package.
How reliably does Balanov/verdict classify real CLI outputs?
Issue #1 records an HTML HTTP error labeled success at 0.734204 and a Bun caller-code error labeled blocked at 0.968427. Balanov reproduced both and reported corrections after expanding the training corpus. The README describes Verdict as a research preview, and the thread supplies specific examples of why an integration should compare predictions with known command outcomes rather than treating a probability as the outcome itself.
Can verdict recognize an HTTP failure when curl exits with code zero?
Balanov reports that the HTML HTTP 500 example classifies as failure after merged PR #2 in issue #1. Before that change, the reproduced label was success with confidence 0.734204. The correction added a training family for the HTML form of the error, giving this particular input a documented before-and-after result and a specific explanation for the changed prediction.
How does verdict distinguish a code failure from an environment blocker?
The output contract defines failure as an operation reporting a concrete failure and blocked as permissions, dependencies, network, quota or environment preventing execution. The Bun example crossed that boundary: a caller-code TypeError received blocked. Balanov reports that adding code_readonly_db corrected the example to failure. The thread makes the distinction practical because those categories can lead an agent toward different recovery actions.
Should a successful Cargo command that ran zero tests be classified as unknown?
Issue #1 expected unknown for test result: ok. 0 passed; 0 failed, but the corrected reproduction returned success 0.600991 and unknown 0.391798. Alloevil retracted the initial unknown-at-0.531 observation after repeating the inputs across installations. The reporter's correction is the result to use when discussing this example, rather than the initial probability that the reporter could no longer reproduce.
Does verdict reject weights whose feature hash differs from the installed code?
Version 0.1.1 rejects a feature-hash mismatch, according to Balanov's response in issue #3. The loader compares the declared and expected hashes before reading weights and raises ValueError naming both values. The v0.1.1 release notes identify incompatible-feature-hash rejection as the release change, connecting the reported implementation to an exact published version.
What complete command, exit code, stdout and stderr fields are needed to reproduce a verdict result?
Preserve command, exit_code, stdout and stderr together. Balanov requested complete JSONL fields during the Cargo investigation in issue #1, where the initial observation differed from the reproduction. Saving all four fields keeps the command context and both output streams attached to the numeric exit status, giving a later investigation the same input rather than a shortened account of it.
What we have not tested yet
Our 14 passing tests and successful setup do not measure broad production-trace classification accuracy. The README calls its corpus synthetic and English-biased, and its probabilities are not calibrated confidence for safety-critical decisions. We have not measured throughput, repeated-inference memory use, non-English performance, platform compatibility, middle-of-log errors affected by clipping, or the consequences of wrong labels in an operating agent. The image label also does not establish actual runtime or operating-system versions; those require raw toolchain logs.
Our run did not independently reproduce the maintainer's curl, Bun or artifact-compatibility corrections. Issue #1's closing response confirms the two training corrections without resolving the reproduced Cargo classification, and the reporter found no evidence that a version change caused the earlier discrepancy. Matching feature hashes establishes compatibility with the feature recipe rather than prediction correctness. The failed registry attempt establishes that command's resolver outcome, rather than a failure of the README's source-install path.
The unrelated Reddit, X, YouTube and Bilibili results concern other uses of the name and supply no adoption evidence for Balanov/verdict. The Variety result returned an access error, and Capterra returned a connection check. Our dependency audit reported zero known vulnerabilities, but this review includes no application security assessment, rendered-layout assessment or test of recommendation modules. The next useful evaluation is a collection of complete command inputs with independently specified expected outcomes.


