mrkeyoor.com_
Wed 04 Nov 10:36 UTC
AI Toolsevaluationupdated 31 Oct 2026

supervision review

Choose Roboflow Supervision when you already have a computer-vision model and need Python utilities around its predictions. Supervision provides model connectors, annotators and dataset utilities. Our run on October 10, 2026, at commit 0802a0e installed and built successfully, but its test command failed. That result supports trying the library with your own inputs; it does not establish deployment readiness or correct evaluation results.

+49stars / 7d
Verdict

Use Supervision when model connectors, annotations and dataset operations solve recurring work in your Python application. Our October 10, 2026 run at commit 0802a0e completed installation and the build step, but the test command failed with dependency-related failures in the raw tail. Reproduce your intended workflow and resolve its optional requirements before adoption; validate evaluation results separately against a reference.

We ran it

Lab card: what happened when we ran supervisionScreenshot of supervision (supervision.roboflow.com)
Install✓ · 26s50 packages · 409 MB
Build✓ · 5s
Tests✗ · 76s0 passed · 0 failed · 2 errors of 2 (pytest)
Known vulns0(pip-audit)
Repo443 files~118,192 lines of source · 71.4 MB · 13 CI workflows · tests dir

Answers from our run

Does supervision build from source?

Dependencies installed in 26 seconds (50 packages), and the build succeeded in 5 seconds. We cloned commit 0802a0e into a clean Debian container with 3 CPUs and no project-specific setup.

Do supervision's tests pass?

No. The project's own test command finished in 76 seconds and reported 0 passed, 0 failed, 2 collection errors (pytest).

Does supervision have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use supervision?

Readers expecting the basic package installation to demonstrate a working detector: the README's prediction example adds Pillow and RF-DETR.

What are the alternatives to supervision?

Norfair, FiftyOne. Use Supervision when model connectors, annotations and dataset operations solve recurring work in your Python application.

Setup3/5Installation and the build step succeeded, but the test command failed with missing optional requirements. The visible rubric assigns 3 to this unresolved setup state.
Docs4/5The README supplies concrete model, annotation and dataset examples. This meets the visible rubric's example-coverage criterion, without proving every workflow is documented.
Community5/5PR #2701 documents contribution, review, revision and merge. The rubric scores that visible process only; adoption, response times and support quality were not measured.
Maturity4/5Release 0.30.9 includes migration guidance and the checkout has tests and CI. The failed sandbox command and untested deployment workload prevent treating this as operational assurance.

Discussed on

  1. hnSupervision: Reusable Computer Vision236 points
  2. hnSupervision – reusable computer vision tools5 points
  3. hnShow HN: Supervision, reusable computer vision utilities5 points

Who it’s for

Python developers who need a common detection representation across model libraries.
Teams loading, splitting, merging or exporting COCO, YOLO and Pascal VOC datasets.
Builders adding annotations or zone-counting workflows around an existing model.

Who it’s NOT for

Readers expecting the basic package installation to demonstrate a working detector: the README's prediction example adds Pillow and RF-DETR.
Teams requiring a clean sandbox test result before adoption: our recorded test command exited unsuccessfully.
Pipelines that must accept negative COCO or CreateML box sizes unchanged: release 0.30.9 rejects them.
Evaluators treating a successful build as evidence that their metrics match a reference implementation.

Setup reality

Our run took place on October 10, 2026, at commit 0802a0e in our Python 3.12 Debian Bookworm sandbox with 3 CPUs and 8 GB RAM. Installation took 26 seconds, added 50 packages and occupied 409 MB. The recorded build step succeeded in 5 seconds; the test step exited with code 1 after 76 seconds.

The attached stored parser summaries record 0 passed, 0 failed, 0 skipped and 2 errors. These are parser results, not an exhaustive count of unique tests. The raw tail contains nested pytest output and a different outer footer; those figures have not been reconciled and are not substituted for the stored summary here.

The raw failures identify missing sklearn during parity-test collection, an OpenCV-blocked integration check whose subprocess could not collect those tests, and metric reporting and aggregation paths requiring the metrics extra. No corrective rerun is attached. Pip-audit recorded 0 advisories, but scan coverage is unknown because scannedPackages is absent. This is not evidence of security or a tested inference deployment.

Should you use Supervision around your detector?

Supervision provides model connectors, annotators and dataset utilities. Its README shows connectors for Ultralytics, Transformers, MMDetection and Roboflow Inference; RF-DETR can return its detection representation directly. The practical reason to consider it is that your application can consume a common representation while using its annotation and dataset operations.

Keep the model boundary in view. The README's RF-DETR prediction example installs Pillow and RF-DETR separately, opens an input image and loads a model. Its Roboflow Inference example requires an API key. Installing Supervision alone does not demonstrate that your chosen model, weights or input stream works.

What our sandbox established

Our run on October 10, 2026, at commit 0802a0e installed and built successfully, but its test command failed. Installation took 26 seconds and left 50 packages occupying 409 MB. The build step returned zero after 5 seconds. The test command returned 1 after 76 seconds, without timing out.

Our harness records step success, exit status, elapsed time and parsed test counts separately. The attached stored parser summaries report 0 passed, 0 failed, 0 skipped and 2 errors. They do not establish exhaustive unique test coverage. In particular, zero parsed failures does not turn a command with an unsuccessful exit status into a passing suite.

The attached raw tail includes an inner pytest invocation and an outer footer with different counts. We have not reconciled the complete log, so we do not promote those footer figures into replacement totals or add them together. Skipped tests are not failures.

The MrKeyoor testing-methodology page explains the distinction between command outcomes, parsed counts and workload coverage. It provides provenance context; its current procedure is not proof that this older run used every current step.

Why the test command failed

The raw tail names three useful failure paths. Collection of tests/metrics/test_sklearn_parity.py could not import sklearn. The OpenCV-blocked integration check failed after its nested pytest process encountered that collection problem. Separately, keypoint metric reporting and aggregate-metric tests raised an ImportError requiring the metrics extra.

These are concrete dependency problems to investigate before reproducing the suite. They do not establish that every failure would disappear after installing optional dependencies, because no corrective rerun is supplied.

The record also does not demonstrate a completed detector inference workload, GPU execution, a video deployment or runtime throughput. A zero build exit status cannot answer those questions. Pip-audit's recorded advisory total is 0, but scannedPackages is absent, so scan coverage is unknown; the audit is not a security assessment.

Check dataset and backend behavior before upgrading

Release 0.30.9 rejects negative COCO and CreateML box sizes and orders reversed Pascal VOC corners. Its migration guidance tells users to correct or remove invalid COCO and CreateML boxes. If your imports previously accepted those labels, check representative files before upgrading.

The release also says its NumPy polygon fallback follows OpenCV 4.13 and later, while older OpenCV uses a different distance rule. Pin the image backend and compare representative polygon exports if downstream work depends on consistent geometry. These are release-note statements, not backend-equivalence measurements from our sandbox.

Evaluation needs a separate acceptance check

The supplied PR page shows that the MeanAverageRecall detection-limit fix merged into develop on October 11, 2026. That is after our recorded run. The PR discussion describes changing the limit from one applied across an image to one applied per image and class, and also identifies a remaining matcher difference from COCO.

The author's parity checks are upstream evidence, not our measurements. The merge does not establish that our measured commit contains the correction or that a particular release matches COCO in every case. If evaluation numbers decide whether a model ships, reproduce the relevant cases against your reference implementation on the exact version you select.

Choose alternatives by the missing job

Norfair is an option when the missing job is multi-object tracking around an existing detector. Its README describes accepting detector outputs expressed as points and inserting tracking into existing video pipelines. That makes it a focused tracking choice, not evidence that it replaces Supervision's dataset utilities.

FiftyOne is an option when the missing job is visual dataset inspection and label editing. Its README describes exploring images, videos and labels in a visual interface and editing labels in the App. It can complement an application using Supervision; choose it for that inspection workflow rather than assuming the two libraries are interchangeable.

How to read the editorial scores

These scores describe the supplied evidence, not popularity, benchmark performance or compatibility across environments. Setup ease uses 1 for a blocked installation, 3 for a completed installation with unresolved test setup, and 5 for a demonstrated clean setup and checks. Documentation uses 1 for no usable entry point, 3 for basic instructions and 4 for concrete examples across the principal workflows; 5 would require broader verified task coverage. Community uses 1 for no visible contribution evidence, 3 for visible participation and 5 for a documented contributor-to-review-to-merge cycle. Maturity uses 1 for an unverified prototype, 3 for documented functionality with checks, 4 for a released project with migration guidance and a test suite, and 5 for stronger verified operational evidence.

On those limited criteria, setup ease is 3, documentation is 4, community is 5 and maturity is 4. The community score reflects the visible contribution process, not adoption or guaranteed support. The maturity score does not override the failed sandbox test command.

Before adopting it, select the operations you need, install their dependencies and check a small set of your own images, labels and expected outputs. For metrics, include reference results rather than relying on a plausible-looking number.

Alternatives

ProjectWhat it isPick it when
NorfairA Python library for customizable multi-object tracking around detector outputs.Choose it when tracking is the missing component in an existing detector pipeline; this is a focused alternative for that job, not a replacement for dataset conversion.
FiftyOneA tool with a visual interface for exploring datasets, editing labels and evaluating models.Choose it for dataset inspection and label review; it can complement Supervision's application utilities.

What people are saying

  1. [github-trending] roboflow/supervision

Sources

  1. Supervision README
  2. Supervision 0.30.9 release
  3. Proposed per-class recall limit fix
  4. Norfair
  5. OpenCV
  6. FiftyOne

More ai tools reviews

copilot-cli · kornia · skills · anarlog · pr-agent · agent-sprite-forge · the whole board →