mrkeyoor.comAsk

dabit3/jev-experimentstested 05 Oct 2026commit c469e5b

jev-experiments review

Use Jev Experiments as a reference for the boundary between model judgment and support policy.

  • Install 22 s
  • Build 13 s
14/14tests passed
  • 141 packages
AI Toolsevaluationupdated 01 Nov 2026
+0stars / 7d

Use Jev Experiments to study how Agent Assist separates model judgments from support policy before evaluating the live integration yourself. Agent Assist uses Jev to select handwritten replies. Our run on October 5, 2026, at commit c469e5b checked installation, build and tests in our sandbox. It did not validate live Jev responses, browser interaction, deployment or support-classification accuracy.

Verdict

Use Jev Experiments as a reference for the boundary between model judgment and support policy. Our sandbox recorded installation in 22 seconds, build in 13 seconds and tests in 7 seconds; those command outcomes did not validate a live support workflow. Start with Agent Assist, evaluate its judgments with approved data, reconcile the audit outputs and establish licensing before adopting its code.

We ran it

Lab card: what happened when we ran jev-experiments
Lab card from our sandbox run
Screenshot of jev-experiments (github.com/dabit3/jev-experiments)
github.com/dabit3/jev-experiments
Install✓ · 22s141 packages · 180 MB
Build✓ · 13s
Tests✓ · 7s14 passed · 0 failed of 14 (vitest)
Known vulns00 critical · 0 high · 0 moderate · 0 low (npm audit)
Repo728 files~59,458 lines of source · 136.4 MB · 1 CI workflows

Answers from our run

Does jev-experiments build from source?

Dependencies installed in 22 seconds (141 packages), and the build succeeded in 13 seconds. We cloned commit c469e5b into a clean Debian container with 3 CPUs and no project-specific setup.

Do jev-experiments's tests pass?

Yes. The project's own test command finished in 7 seconds and reported 14 passed, 0 failed (vitest).

Does jev-experiments have known vulnerabilities in its dependencies?

npm audit found none in the dependency tree at the time of our run.

Who should not use jev-experiments?

Buyers seeking independently measured support accuracy or live response latency: our run established neither.

What are the alternatives to jev-experiments?

AI SDK, NanoJev. Use Jev Experiments as a reference for the boundary between model judgment and support policy.

Setup4/5Strong evidence within the measured Agent Assist setup scope: installation and build succeeded. Live use still requires credentials and separate validation.
Docs4/5Strong evidence within the supplied Agent Assist documentation: request contents, policy rules, confidence handling, mock mode and startup are explained.
Community2/5Last push on 2026-09-21 and 413 stars.Editor’s note: The supplied primary extracts do not establish maintainer responsiveness, sustained contribution activity or support outcomes.
Maturity2/5A limited reference for support-application adoption: local commands succeeded, while live validation, audit reconciliation and licensing remain unresolved.

How we score

Discussed on

  1. hnTypeSafe / Jev latency-focused demos built by Devin3 points

Who it’s for

TypeScript developers studying a concrete Jev integration.
Support-tool builders deciding which judgments belong in a model and which rules belong in application code.
Teams able to evaluate the live service with approved conversation data and a TypeSafe API key.

Who it’s NOT for

Buyers seeking independently measured support accuracy or live response latency: our run established neither.
Teams requiring an offline support model: the documented live path calls TypeSafe; mock mode returns canned keyword answers.
Developers expecting one application at the repository root: the repository describes separate applications in top-level directories.
Teams ready to adopt code without resolving licensing and the conflicting recorded audit outputs.

Setup reality

Our run on October 5, 2026, at commit c469e5b used agent-assist in our sandbox with 3 CPUs, 8 GB RAM and the lab-node:22 image. The harness recorded installation in 22 seconds, build in 13 seconds and tests in 7 seconds. Each command returned exit code 0 without a timeout.

The stored parser summary reports 14 passed, 0 failed and 0 skipped tests. These counts do not establish exhaustive unique coverage. The raw test tail repeats its summary; repeated lines are not additional tests.

The stored dependency measurement records 141 packages and 180 MB. This is a dependency-footprint measurement, not an audited-package coverage count. Scan coverage is unknown because scannedPackages is absent. The stored audit summary records zero advisories, while the raw installation output reports two critical-severity vulnerabilities; the supplied record does not reconcile them.

The Agent Assist README requires Node 22+ and a TypeSafe API key for live use. The development command starts a Node proxy on port 8787 and Vite on port 5173. MOCK=1 supplies labelled canned answers for offline interface work and does not evaluate Jev.

Our sandbox completed local commands, not a live support session

Our run on October 5, 2026, at commit c469e5b recorded a successful Agent Assist build in 13 seconds. The attached harness records installation separately at 22 seconds and the test command at 7 seconds. These are wall-clock command durations in our sandbox, not model-response times. The build tail shows TypeScript and Vite completing; it does not establish that a browser session, hosted request or deployment worked.

Our harness records command outcomes, exit codes, timeout flags, elapsed time and parsed test summaries. The stored parser summary reports 14 passed, 0 failed and 0 skipped tests. The raw tail names engine.test.ts and questions.test.ts, then repeats the final totals. We have not reconciled complete logs or verified exhaustive unique coverage. The successful commands provide evidence for evaluating this subproject locally, without establishing what every test covers or whether other demos work.

The MrKeyoor testing methodology explains how command records inform reviews. The supplied methodology page was fetched on October 31, 2026. It provides provenance context, but does not prove that its current procedure governed our October 5 run. This review uses the stored run and adds no new execution.

Agent Assist selects replies while code enforces policy

Agent Assist uses Jev to select handwritten replies. The Agent Assist README describes a support console where each incoming customer message triggers one request containing questions about intent, churn risk, frustration, flags and macro selection. Refund eligibility remains a TypeScript calculation based on plan and tenure. This division gives a developer a concrete boundary to inspect: the model judges language, and application code decides how to act on its output.

The confidence threshold is part of that boundary. At confidence 0.6 or above, the selected macro is auto-filled; below it, the interface presents candidates for the agent to choose. A none result means the agent writes a reply. The documented hands-free path can auto-send an auto-filled reply 1.5 seconds later. Before adapting that behavior, decide which actions require confirmation and test cases where a plausible suggestion would violate policy.

The Jev Experiments introduction describes separate applications with their own documentation. Start with Agent Assist if support workflows are your question. Our sandbox evidence applies to that subdirectory and supplies no build or runtime result for the other applications.

Live evaluation requires credentials and approved conversation data

The documented live request sends the last six conversation messages, customer plan, tenure, prior-ticket count and macro summaries to TypeSafe. According to the README, the API key stays in the local server and never reaches the browser bundle. Keeping the key server-side does not keep conversation text local. Evaluate this path with data you are permitted to send to the service; use mock mode for interface work that does not require model judgments.

The README reports a 93 ms median panel refresh from its own scripted live run and describes hand-checked judgments on seeded conversations. Those are upstream observations, not our measurements. Its four-second language-model baseline is explicitly simulated. Our run therefore establishes neither that latency nor a measured speed advantage over another deployed copilot. A useful next evaluation should record answer quality separately from panel-update time, using your account, network and cases where mistakes carry a cost.

The recorded audit outputs conflict

The stored audit summary records zero advisories. Separately, the raw installation output says "2 critical severity vulnerabilities". Neither output explains the disagreement, and the installation message does not identify affected packages or establish exploitability. Both remain part of the record. A zero summary cannot support a clean security conclusion when the supplied raw output conflicts with it.

Scan coverage is unknown because the record has no scannedPackages field. The stored dependency count measures footprint and cannot fill that gap. Obtain a current audit report identifying the dependency tree and affected packages before adopting code. That would be a separate check; this correction does not claim that one occurred. The stored license signal is also null, which leaves licensing unresolved rather than establishing a legal conclusion.

Alternatives address different parts of the application

AI SDK provides a unified API for multiple model providers. Its repository documents text generation, structured output, agents and UI integration. Evaluate it when generated responses or provider choice are central to the application. It is an application toolkit that could complement a decision-oriented design; the supplied evidence does not establish Jev integration or a drop-in replacement for Agent Assist.

NanoJev makes its model and dataset publicly downloadable without signing in. Its repository also supplies local inference instructions for a CUDA environment and reports game evaluations. Evaluate it when inspecting or operating a decision model is the goal. This is a separate model evaluation: the supplied game results do not establish support-triage accuracy or compatibility with Agent Assist.

Ratings apply to the evidence available here

The five-point rubric is: 1 for demonstrated obstacles that prevent the intended evaluation, 2 for a limited reference requiring substantial adoption work, 3 for adequate evidence with material gaps, 4 for strong evidence within the stated scope, and 5 for broader corroboration across that scope. Unknown dimensions receive no numerical rating. These are editorial assessments, not measured product benchmarks.

Setup ease is 4 for the measured subproject because installation and build succeeded, with live setup still requiring credentials and validation. Documentation is 4 for the supplied Agent Assist instructions covering requests, policy rules, confidence handling, mock mode and startup. Community remains unassessed because the supplied primary extracts do not establish maintainer responsiveness or sustained contribution activity. Maturity is 2 for adoption as a support application: local commands succeeded, but live workflow validation, audit reconciliation and licensing remain unresolved.

Alternatives

ProjectWhat it isPick it when
AI SDK gh↗A provider-agnostic TypeScript toolkit documenting text generation, structured output, agents and UI integration.Evaluate it when generated responses or provider choice are central. It may complement the application design; the supplied evidence does not establish a drop-in Jev replacement.
NanoJev gh↗A decision-model project with public model and dataset downloads and local CUDA inference instructions.Evaluate it when model inspection or local operation matters. Its supplied game results do not establish support accuracy or Agent Assist compatibility.

What people are saying

  1. [velocity-scout] dabit3/jev-experiments

Sources

  1. Jev Experiments repository
  2. Agent Assist README
  3. Agent Assist package manifest
  4. Say-only GitHub Actions workflow
  5. Open adversarial policy probes pull request

More ai tools reviews