mrkeyoor.com_
Tue 03 Nov 01:44 UTC
AI Toolsevaluationupdated 31 Oct 2026

ADR review

Choose Uber ADR for agent discovery, telemetry collection or detector research when your team can govern sensitive records and isolate the benchmark. ADR Prevention is excluded from the public release. ADR means Agentic AI Detection and Response; its public components include Discovery, Sensor, ADR-Bench and Detection. Our run on September 27, 2026, at commit abc12c2 covered installation, build and tests in Detection/, not a live agent deployment or detector accuracy evaluation.

+8stars / 7d
Verdict

ADR fits a controlled agent-observability or detector-research project. Our September 27, 2026 sandbox run at abc12c2 recorded successful Detection install, build and test commands, with a stored parser summary of 248 passed and no recorded failures or errors. It did not validate a deployment or detector effectiveness. Start by selecting the component you need, defining its data boundary and isolating benchmark work. Teams that require prevention need a separate enforcement control.

We ran it

Lab card: what happened when we ran ADRScreenshot of ADR (arxiv.org/abs/2605.17380)
Install✓ · 154s324 packages · 7037 MB
Build✓ · 5s
Tests✓ · 42s248 passed · 0 failed of 248 (pytest)
Known vulns166(pip-audit)
Repo565 files~89,649 lines of source · 34.6 MB · 4 CI workflows · tests dir

Answers from our run

Does ADR build from source?

Dependencies installed in 154 seconds (324 packages), and the build succeeded in 5 seconds. We cloned commit abc12c2 into a clean Debian container with 3 CPUs and no project-specific setup.

Do ADR's tests pass?

Yes. The project's own test command finished in 42 seconds and reported 248 passed, 0 failed (pytest).

Does ADR have known vulnerabilities in its dependencies?

pip-audit flagged 166 known advisories in the dependency tree at the time of our run.

Who should not use ADR?

Teams requiring the public release to prevent unsafe agent actions.

What are the alternatives to ADR?

garak, promptfoo. ADR fits a controlled agent-observability or detector-research project.

Setup2/5Substantial operator work: our sandbox recorded 324 packages and 7,037 MB, while the documentation requires benchmark isolation and model access for the default detector.
Docs5/5Unusually explicit guidance within the reviewed scope: the READMEs explain setup, isolation, sensitive telemetry, capture limitations and paper-only baselines. Documented paths were not all executed.
CommunityN/AThe supplied primary extracts do not establish contributor breadth, maintainer response times or support quality; attention figures cannot establish those dimensions.
Maturity3/5Usable documented scope with material limits: public discovery, collection and detection components exist, while Prevention and Explorer are excluded. Production use at Uber is a vendor statement.

Discussed on

  1. hnUber open-sourced its security monitoring for Claude Code, Cursor and Codex3 points

Who it’s for

Enterprise security engineers investigating which AI tools are present on employee endpoints.
Detection engineers collecting normalized agent records for downstream analysis.
Researchers evaluating detectors against synthetic scenarios in an isolated environment.
Teams with approved collection, access, retention and export policies for sensitive agent telemetry.

Who it’s NOT for

Teams requiring the public release to prevent unsafe agent actions.
Operators planning to connect the research benchmark to production networks, real credentials or live MCP servers.
Buyers who need our run to establish detector effectiveness, complete telemetry capture or production readiness.
Researchers expecting runnable ALRPHFS and GuardAgent baselines; the Detection README identifies their results as paper-only numbers.

Setup reality

Our sandbox's September 27, 2026 run at commit abc12c2 recorded installation in 154 seconds, build in 5 seconds and tests in 42 seconds within Detection/. Each command returned exit code 0 without a timeout. The recorded dependency footprint was 324 packages and 7,037 MB.

The stored parser summary records 248 passed, 0 failed, 0 skipped and 0 errors. These counts do not establish exhaustive unique test coverage. The raw pytest footer reports "248 passed, 25 warnings in 28.27s"; its duration is separate from the harness's 42-second test-step duration. The supplied tail contains Pydantic deprecation warnings and a LangChain community syntax warning.

Pip-audit recorded 166 advisories across 19 packages. Total scan coverage is unknown because the record has no scannedPackages value. Advisories do not establish exploitability or constitute a security review.

The documented default detector requires OpenAI and Anthropic API keys, Claude Code and an authenticated Claude session. Hugging Face access is optional. A keyless LlamaFirewall smoke-test path is documented, but our record does not demonstrate execution of that path.

What did our sandbox establish?

Our sandbox recorded successful install, build and test commands in Detection/ on September 27, 2026, at commit abc12c2. The full revision was abc12c20893f793aa1c22fcd0296052423de00f1. Installation took 154 seconds, with 324 packages occupying 7,037 MB; build took 5 seconds and the test step took 42 seconds. These observations belong to that stored run. This correction involved no new execution.

The stored parser summary records 248 passed, 0 failed, 0 skipped and 0 errors. It does not establish exhaustive unique test coverage. The raw pytest footer separately reports "248 passed, 25 warnings in 28.27s". We retain that suite duration alongside the harness's command duration and do not add repeated footer counts. The attached run records no collection error.

Our harness records command outcomes through success status, exit code, elapsed time, timeout status and parsed test counts. Each of these three commands returned zero without timing out. The MrKeyoor testing methodology explains this reporting model and its limits; its current description does not establish which methodology version governed this older run. The record does not demonstrate live agent collection, a Sensor CLI deployment, OTLP delivery, benchmark workload completion or detector accuracy.

Does the public release prevent unsafe actions?

ADR Prevention is excluded from the public release. The Uber ADR README lists Discovery, Sensor, ADR-Bench and Detection as public components and also excludes the offline Explorer engine. Discovery inventories endpoint AI tools; Sensor normalizes recorded activity; the benchmark supplies evaluation material; Detection combines triage with deeper reasoning about suspicious sessions.

Use those components when investigation or evaluation is your immediate need, and plan enforcement separately. Uber reports deploying ADR in production, but that vendor statement does not validate deployment of the public components in your environment. It also does not extend our Detection command results to the Sensor or Discovery components.

Can you deploy the benchmark in production?

The Detection README requires an isolated environment and says the benchmark is not intended for production deployment. It explains that dependencies pinned for paper reproducibility carry known CVEs. Synthetic credentials, prompt-injection payloads and emulated vulnerable MCP servers are intentional research fixtures; the documentation forbids pointing the benchmark at live systems, real credentials or production MCP servers.

Our sandbox's pip-audit result recorded 166 advisories across 19 packages. These are recorded dependency advisories, not demonstrated exploits. The record does not supply scannedPackages, so the total number of packages scanned remains unknown. Successful tests do not remove the Detection README's isolation requirement or establish the security of the dependency tree.

What data can Sensor export?

The Sensor's documented OTLP exporter applies no redaction or field projection to captured telemetry. Its payload can include prompts, responses, tool arguments, tool results, usernames, hostnames and local paths. Export is disabled by default and requires an explicit configuration file. Before enabling collection or export, decide which records your organization may retain, who can access them and which destination may receive them.

The Sensor README also describes source-specific limits. Claude Code arguments and results have documented truncation, while the Gemini parser adds no redaction or truncation. Normalization therefore does not establish identical capture across agents. Content-free health diagnostics are separate from captured session data, and the documentation warns that a healthy summary does not prove complete capture. Our Detection run did not test these endpoint behaviors.

Did our tests validate detector effectiveness?

Our run did not measure detection precision, recall or false positives. The README's paper results concern a particular evaluation, while our stored parser summary describes test-command output. Treating those as the same evidence would overstate what we tested.

The Detection documentation distinguishes the current task inventory from the paper's evaluation set. It also identifies ALRPHFS and GuardAgent as paper-only results because their implementations were removed for licensing reasons. Before attempting reproduction, check the available conversations, labels and runnable detector. The default ADR path requires hosted-model access and Claude CLI authentication; the documented keyless LlamaFirewall path offers a narrower smoke test, without evidence here that our sandbox executed it.

Which alternative fits application testing?

Garak probes LLM-backed applications for unwanted behavior. Its README describes probes for hallucination, data leakage, prompt injection and other weaknesses. Choose garak when probing an application or model is the immediate task. That role can complement ADR's collection of recorded agent activity; the supplied evidence does not establish equivalent endpoint discovery or telemetry collection.

Promptfoo is a CLI and library for evaluating and red-teaming LLM-based applications. Its README also documents automated CI/CD checks. Choose promptfoo for application evaluations within a development workflow. It can complement ADR telemetry rather than replace the Sensor's collection role. Neither alternative was executed in our ADR sandbox run.

What do the editorial ratings mean?

Setup ease is 2/5 because the measured footprint and documented isolation and authentication requirements demand substantial preparation. These ratings assess the reviewed evidence, not runtime performance or security. The rubric is: 1 means major obstacles, 2 substantial operator work, 3 usable scope with material limits, 4 strong documented support and 5 unusually explicit guidance within the reviewed scope. A dimension without sufficient evidence remains unassessed.

Documentation receives 5/5 for explicit setup, isolation, telemetry and reproduction boundaries, without implying every path was tested. Maturity receives 3/5 for documented public components with material exclusions; Uber's production statement remains vendor evidence. Community is unassessed because the supplied extracts do not establish contributor breadth, maintainer responsiveness or support quality.

Alternatives

ProjectWhat it isPick it when
garak gh↗A scanner that probes LLM-backed applications for unwanted behavior.Choose it for model or application probing. It can complement ADR telemetry; the supplied evidence does not establish it as a replacement for endpoint discovery or Sensor collection.
promptfoo gh↗A CLI and library for evaluating and red-teaming LLM-based applications.Choose it for application evaluations and automated CI/CD checks. Use it alongside ADR when you also need recorded endpoint agent activity.

What people are saying

  1. [github-trending] uber/ADR

Sources

  1. Uber ADR README
  2. ADR Detection README
  3. ADR Sensor README
  4. ADR reproducibility guide
  5. ADR Sensor v1.0.0 release
  6. Unicode obfuscation issue 42

More ai tools reviews

kornia · skills · anarlog · pr-agent · agent-sprite-forge · KiroCrew · the whole board →