Choose the installation route before the plugin
wshobson/agents distributes reusable instructions for coding assistants. The practical question is whether your chosen installation route carries the components you need. A source skill, a generated agent profile, and a Claude Code plugin can expose different behavior even when they originate in the same repository.
The current README lists 94 plugins, including 92 local plugins and 2 external entries. It reports 202 agents, 185 skills, and 105 commands in the local source. These are repository claims, not counts verified by our sandbox. The README at measured commit 156b7a5 instead listed 184 skills and 16 orchestrators. That historical inventory should not be presented as the current catalog.
gh skill and npx skills install skills without a clone or generation step. They omit agents, commands, and hooks. Claude Code loads the source plugin format through its marketplace. Codex and Cursor have committed marketplace registries, but their component support differs. OpenCode, Antigravity CLI, GitHub Copilot, and Pi use generated artifacts from a clone.
Shared source does not mean identical behavior
The harness guide distinguishes native Codex installation from generated Codex artifacts. Native manifests expose source skills. Generated TOML agents and commands converted into skills use a separate route. Native skill bodies over the 8 KB cap are truncated by Codex at load; generated copies split oversized bodies into reference files. Installing from the native marketplace does not automatically provide that split.
Permissions also change. Codex replaces exact agent tool allowlists with a coarser sandbox_mode choice. Cursor does not preserve those allowlists. OpenCode translates them into permission blocks. Pi agents require its reference subagent extension or a compatible extension. Commands can become skills or prompt templates, and lifecycle-hook support varies.
Several transformed trees are gitignored and must be regenerated locally with make generate. Review those files as part of adopting a plugin. A common source can reduce repeated writing, but each coding tool remains a separate integration to check.
What our recorded run establishes
Our run began on September 29, 2026 at 06:03:05.082 UTC and used commit 156b7a5e7a8b93642628a339ee4039c925b34c7f. Its detected project was plugins/plugin-eval/.
The dependency-install step succeeded in 36.907 seconds, rounded to 37 seconds in the stored summary, with 47 packages and 100 MB recorded. The build step succeeded in 8.425 seconds, rounded to 8 seconds. Its retained output is empty, so the record does not identify a build artifact or establish an installed-binary check.
The test step completed in 14.944 seconds, rounded to 15 seconds. The retained pytest output reports "141 passed, 2 skipped in 4.10s"; the parser records 0 failures and 0 errors. The pytest duration and enclosing step duration measure different intervals. The stored total of 141 must not be used to describe the skipped tests as a subset of 141 exhaustive cases. No complete-log reconciliation or new execution verified unique, exhaustive coverage.
The install output identifies Python 3.12.12. The recorded configuration specifies 3 CPUs, 8 GB of memory, and image ghcr.io/astral-sh/uv:python3.12-bookworm. The checkout summary records 1,190 files, approximately 21,303 lines of source, and 8.2 MB. The scan found 6 CI workflow files and a tests directory, with no Dockerfile or Compose file detected. Total recorded run time was 78 seconds.
Pip-audit recorded 0 advisories. The record does not establish which packages were inspected or the audit's coverage. Neither the 47-package dependency figure nor a zero advisory total supplies that missing evidence.
These observations concern the evaluator subproject. They do not establish that a PluginEval CLI binary was installed and invoked, that any marketplace plugin completed a task, or that adapters behaved correctly in their target tools. The run contains no measured agent workload, output-quality comparison, model-cost benchmark, or task-completion result.
Read PluginEval badges as evaluation outputs
PluginEval's documentation describes its quick layer as deterministic static lint with no model calls. It checks structure such as frontmatter, headings, reference links, and portability. It does not execute a skill to see whether an agent completes a task. Plugin-directory scores and weekly plugin badges use this static layer alone, regardless of the requested depth.
For individual skills, the documented standard depth uses 4 model calls. Deep depth uses 55, and thorough uses 105. The judge and Monte Carlo layers are experimental and have not been validated against human labels. Monte Carlo activation counts any non-empty reply, its quality measure uses reply length, and its 15 prompts repeat to fill 50 or 100 runs. Those measures cannot establish that an answer is correct.
The core evaluator requires Python 3.12 or later and uv. Optional model layers use the Claude CLI through additional dependencies. When ANTHROPIC_API_KEY is set, calls bill that API account; otherwise they use the Claude Code login. These are documented requirements and behavior, not model calls exercised by our recorded run.
Pin a trial and check a known task
The supplied releases page contains no releases. The harness guide says gh skill therefore tracks main unless a commit is pinned with --pin. Use a fixed commit when you need a reproducible trial.
Issue 742 was opened on September 28, 2026 and closed through pull request 743 on September 29. The measured commit removes dangling reference pointers and adds a gardener check for that pattern. This is a concrete maintenance example, not evidence of a general response-time guarantee or broad adoption. The supplied extracts do not substantiate the old star count, open-queue count, or last-push claims, so those figures do not inform this review.
Start with one plugin for a recurring task. Check which components your installation actually exposes, inspect permission changes, and try a case with a known expected result. Use PluginEval lint to identify structural problems, then assess task behavior separately.
How to read our scores
Scores are editorial judgments about the supplied evidence, not benchmark results. The scale is 1 for documented major obstacles, 2 for limited evidence or substantial constraints, 3 for a documented usable route with material limits, 4 for detailed supporting evidence within a stated scope, and 5 for broad corroboration across the workflows being assessed. Missing evidence limits a score; it does not prove poor product quality.
Setup receives 3/5 because installation routes are documented but differ materially and were not exercised across tools. Documentation receives 4/5 because the supplied guides explain installation, component differences, and evaluator limitations. Community receives 2/5 because one documented issue and fix do not establish adoption or sustained support. Maturity receives 3/5 because the evaluator's recorded steps succeeded and structural checks are documented, while tagged releases and measured task-level compatibility remain absent from this evidence.