mrkeyoor.comAsk

ekzhang/openjev-sglangtested 05 Oct 2026commit bf6a53b

openjev-sglang review

Evaluate SGLang's native decisions API for a new service.

  • Install 34 s
  • Build 6 s
48/77passed, run exited non-zero
  • 71 packages
LLM Toolsevaluationupdated 31 Oct 2026
+0stars / 7d

For a new decision service, start by evaluating SGLang's native API: the openjev-sglang maintainer recommends it instead of this experiment. openjev-sglang implements the TypeSafe/Jev HTTP API using Qwen3.6-35B-A3B on SGLang 0.5.19. It remains useful as a reference for first-token decision scoring or for maintaining an existing endpoint.

Verdict

Evaluate SGLang's native decisions API for a new service. Keep openjev-sglang as a reference or maintain it while checking an existing client's migration requirements. Our sandbox recorded successful install and build commands and a failing test command, with async-plugin problems visible in the tail. It did not establish working GPU inference, deployment readiness, decision accuracy, or a successful migration.

We ran it

Lab card: what happened when we ran openjev-sglang
Lab card from our sandbox run
Screenshot of openjev-sglang (ekzhang--openjev-sglang-openjev.us-west.modal.direct)
ekzhang--openjev-sglang-openjev.us-west.modal.direct
Install✓ · 34s71 packages · 218 MB
Build✓ · 6s
Tests✗ · 13s48 passed · 3 failed · 26 errors of 77 (pytest)
Known vulns0(pip-audit)
Repo83 files~5,014 lines of source · 2.1 MB · 0 CI workflows · tests dir

Answers from our run

Does openjev-sglang build from source?

Dependencies installed in 34 seconds (71 packages), and the build succeeded in 6 seconds. We cloned commit bf6a53b into a clean Debian container with 3 CPUs and no project-specific setup.

Do openjev-sglang's tests pass?

No. The project's own test command finished in 13 seconds and reported 48 passed, 3 failed, 26 collection errors (pytest).

Does openjev-sglang have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use openjev-sglang?

New adopters looking for the maintainer's recommended implementation.

What are the alternatives to openjev-sglang?

SGLang, vLLM. Evaluate SGLang's native decisions API for a new service.

Setup2/5Local installation completed, but actual inference requires a separate suitable SGLang backend; the supplied Modal profile requires a B200.
Docs5/5The supplied README explains request shapes, inference, probability limits, caching, configuration, and deployment. This assesses documentation coverage, not verified runtime behavior.
Community2/5The supplied repository snapshot gives limited evidence of sustained support. Attention counts do not establish adoption, maintainer responsiveness, or compatibility.
Maturity1/5The archived project describes itself as an early experiment and recommends SGLang instead. Our test command failed, with async-plugin problems visible in the supplied tail.

Who it’s for

Engineers maintaining an existing openjev-sglang client or deployment.
Developers studying shared-prefix inference and answer-label probabilities.
Operators who can evaluate the documented Modal and B200 deployment separately from local API setup.

Who it’s NOT for

New adopters looking for the maintainer's recommended implementation.
Teams treating our sandbox build result as evidence of working GPU inference or a successful Modal deployment.
Applications requiring image, audio, or video input through this repository's documented chat state.
Operators expecting authentication and an explicit container cap in the supplied Modal defaults.

Setup reality

Our run on October 5, 2026 used commit bf6a53b. In our sandbox, the harness recorded install success in 34 seconds, build success in 6 seconds, and a test command failure in 13 seconds. Installation added 71 packages with a recorded footprint of 218 MB.

The attached stored parser summaries report 48 passed, 3 failed, 26 errors, a total field of 77, and zero skipped tests. These fields do not establish exhaustive unique test coverage. The raw footer separately reports one deselected test; deselection is not a failure.

The record does not establish that we ran the Modal deployment, the live inference smoke command, or a GPU workload. The README separates local API dependencies from SGLang's CUDA environment. Its supplied deployment assigns one B200 per container, downloads weights on the first GPU start, and captures kernels at startup.

Should you adopt it or move to SGLang?

The openjev-sglang maintainer recommends SGLang's native decisions API instead of this experiment. That is the strongest reason to evaluate SGLang first for a new service. The supplied GitHub metadata also marks openjev-sglang as archived. Existing users should check the native API's compatibility details before changing their endpoint.

This repository still has a useful, narrower role: explaining how a Jev-compatible API can turn model scores into bounded decisions. Its documented implementation uses Qwen3.6-35B-A3B with SGLang 0.5.19. Noul returns a yes probability, Choice returns the most probable option and its distribution, and Score returns a probability-weighted level index.

What our sandbox established

Our run on October 5, 2026 used commit bf6a53b. The matching sandbox record specifies 3 CPUs, 8 GB of memory, and the ghcr.io/astral-sh/uv:python3.12-bookworm image. Its install tail identifies Python 3.12.12.

Our harness records each command's success flag, exit code, elapsed time, timeout status, and available log tail. Here, install and build returned exit code 0; the test command returned exit code 1. The attached harness durations are 34 seconds for installation, 6 seconds for build, and 13 seconds for tests. These are command outcomes, not evidence that a deployed decision service worked.

The stored parser summaries report 48 passed, 3 failed, and 26 errors, with a total field of 77 and zero skipped tests. They are not verified exhaustive counts of unique tests. We have not performed a new execution or reconciled a complete log. The raw pytest footer separately reports one deselected test and a pytest duration of 5.40 seconds; that footer duration differs from the harness's elapsed command time.

The three named backend failures concern sibling cancellation, the global branch concurrency limit, and sharing a selected-logprob batch between warmup and scoring. Each says async functions are not natively supported. The tail also warns that asyncio_mode is an unknown configuration option. An API test reports an async fixture without a handling plugin; other API entries show AssertionError without enough traceback to diagnose their causes. These observations warrant checking the test environment before interpreting the result as broken inference behavior.

The build tail is empty. Its zero exit code does not establish packaging correctness, CLI behavior, GPU execution, or Modal deployment. No live inference result is attached.

The recorded pip-audit advisory total is zero. Scan coverage is unknown because no scannedPackages value is supplied; the separate packages field is zero and does not establish dependency coverage. This is not a security assessment.

The supplied testing methodology explains the publisher's command recording and sandbox limits. That page was fetched on October 31, 2026; it does not establish which methodology version governed our October 5 run.

What the decision probabilities mean

The server warms a common prefix, then requests one token for each question and reads the answer labels' log probabilities. It ignores the sampled token and normalizes the requested label scores. The documented implementation makes N+1 one-token calls for N questions, including the warmup.

That mechanism avoids generating and parsing a prose answer, but it does not establish decision accuracy or a cost advantage for your workload. The README says option order and prompt wording affect the probabilities. Its confidence value describes the distribution's concentration rather than calibrated correctness.

The documented chat state accepts text, not image, audio, or video content. Cache reuse is opportunistic. SGLang 0.5.19's Rust frontend omits cache counts, so the API leaves its cache-count header absent rather than reporting zero. Neither the README's live-deployment observations nor its smoke-test description are measurements from our sandbox.

What deployment still requires

Local dependency installation covers the API, deployment tools, and tests. Real inference requires a suitable SGLang backend with matching model and tokenizer revisions, selected-token log probabilities, radix caching, and enough context capacity.

The included Modal profile assigns one B200 per container. Initial startup downloads model weights and compiles or captures kernels; persistent volumes retain weights and compilation caches, while CUDA graph capture still runs at startup. A server waking from zero returns 503 while starting.

The sample Modal endpoint is intentionally unauthenticated and has no explicit container cap. API and backend bearer keys are optional. Before exposing that configuration, choose authentication and capacity settings appropriate to your service. Our recorded scan found a tests directory, no CI workflow files, and no Dockerfile; that scan does not establish the absence of checks outside the checkout.

Which alternative answers your requirement?

SGLang's decision-model documentation describes both a native decisions route and a System One compatible route. This makes it the relevant replacement candidate for this repository. The documentation says to use a nightly build until a release contains the endpoint, so verify availability in the version you intend to deploy.

Compatibility still needs testing. The documented System One route has exceptions, including unsupported SDK model listing and response request IDs, different usage accounting, and confidence thresholds that may not transfer between providers. Validate representative requests and thresholds before moving an existing client.

vLLM documents an OpenAI-compatible API server. Consider it when your requirement is general model inference and serving. The supplied vLLM extract does not establish a Jev-compatible decisions endpoint, so it does not support treating vLLM as a direct replacement for this API. Neither alternative was executed in our recorded run.

How to read the ratings

These editorial ratings use a five-point scale: 1 indicates a substantial constraint, 2 requires significant operator work or has limited supporting evidence, 3 indicates adequate evidence for ordinary use, 4 indicates strong evidence with minor gaps, and 5 indicates unusually complete evidence for the dimension assessed. They are judgments, not benchmark results.

Setup ease remains 2 because installation completed but inference needs a separate backend and the supplied deployment requires GPU infrastructure. Documentation remains 5 for its detailed explanation of requests, probability semantics, caching, configuration, and deployment limits; that rating does not verify every documented behavior.

Community remains 2 because the supplied snapshot provides limited evidence of sustained support. Stars and issue counts cannot establish adoption or response quality. Maturity remains 1 because the project calls itself an early experiment, recommends another implementation, is marked archived, and did not complete the recorded test command successfully.

Alternatives

ProjectWhat it isPick it when
SGLang gh↗The maintainer-recommended replacement candidate, with documented native decisions and System One compatible routes.Evaluate it for a new decision service or an existing Jev-compatible client's migration. Check endpoint availability in your selected version and test the documented compatibility exceptions.
vLLM gh↗An inference and serving library with a documented OpenAI-compatible API server.Consider it for general model serving. The supplied evidence does not establish it as a direct replacement for this Jev-compatible decision API.

What people are saying

  1. [velocity-scout] ekzhang/openjev-sglang

Sources

  1. openjev-sglang README
  2. openjev-sglang repository facts
  3. SGLang decision models documentation
  4. openjev-sglang issues and pull requests

More llm tools reviews