mrkeyoor.com_
Thu 17 Sept 16:00 UTC
AI Toolsevaluationupdated 17 Sept 2026

skills review

Trail of Bits Skills is a marketplace of 44 Claude Code plugins for security review, vulnerability research, verification, and development work. Codex can load the same marketplace, and selected skills include metadata for ChatGPT workspace imports.

Verdict

Our constant-time-analysis run installed 36 packages in 19 seconds and built in 1 second, but that project exposed no test target for us to run. Security teams using Claude Code or Codex should browse this marketplace because its 44 plugins encode much more specific review work than a generic security prompt. Install only the plugins you can supervise, confirm their external tools, and run the repository's wider checks before modifying or redistributing them.

We ran it

Lab card: what happened when we ran skillsScreenshot of skills (github.com/trailofbits/skills)
Install✓ · 19s36 packages · 37 MB
Build✓ · 1s
Testsn/ano test script
Known vulns0(pip-audit)
Repo1441 files~92,367 lines of source · 8.7 MB · 3 CI workflows

Answers from our run

Does skills build from source?

Dependencies installed in 19 seconds (36 packages), and the build succeeded in 1 seconds. We cloned commit 123037e into a clean Debian container with 3 CPUs and no project-specific setup.

Does skills have tests you can run?

Not through a standard command: the project exposes no test script or target that our harness could run.

Does skills have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use skills?

Teams expecting an agent skill to certify code as safe: the constant-time plugin labels branch findings for review, and its skill adds human judgment around analyzer output.

What are the alternatives to skills?

Anthropic Skills, Sentry Skills, Superpowers. Our constant-time-analysis run installed 36 packages in 19 seconds and built in 1 second, but that project exposed no test target for us to run.

Setup3/519-second scoped install; full checks need several system tools
Docs5/5Plugin catalog, author rules, dependencies, and check gaps are explicit
Community4/57,127 stars with push and pull request activity in September 2026
Maturity3/5Strong validation rules, but plugin-specific failures remain open

Who it’s for

Security engineers who want repeatable agent instructions for code audits, smart contracts, malware, and reverse engineering.
Claude Code or Codex users who prefer installing narrowly scoped plugins instead of one general prompt file.
Teams that already use tools such as Semgrep, CodeQL, compilers, fuzzers, or SARIF in their review process.
Skill authors who want working examples with scripts, references, hooks, agents, and validation rules.

Who it’s NOT for

Teams expecting an agent skill to certify code as safe: the constant-time plugin labels branch findings for review, and its skill adds human judgment around analyzer output.
Cursor or Gemini CLI users who need documented support now: compatibility requests for both remain open, while the README names Claude Code, Codex, and ChatGPT workspace imports.
Windows teams that cannot tolerate plugin-specific breakage: issue 289 reports CRLF-damaged packaged workflows and an fp-check hook deadlock.
Organizations that require a permissive license for redistributed skill text: the repository uses CC BY-SA 4.0.
Contributors seeking one small, self-contained toolchain: the root checks require uv, shellcheck, shfmt, bats, and Node, while individual security plugins can need their own compilers or scanners.

Setup reality

Our sandbox scoped the runnable project to plugins/constant-time-analysis/. It installed 36 Python packages in 19 seconds, used 37 MB, and built in 1 second. No test script or target was declared for that project, so tests were skipped. Pip-audit found 0 known vulnerabilities.

Marketplace use needs Claude Code, Codex, or a compatible ChatGPT workspace import. The constant-time analyzer requires Python 3.10 or newer and uv; analysis of C, Rust, Go, JavaScript, and other targets also depends on the relevant compiler or runtime.

The root make check needs uv, shellcheck, shfmt, bats, and Node. It still excludes Claude and Codex loadability checks plus Chromium browser checks. The repository has no Dockerfile, so contributors assemble those tools on the host or in their own image.

Forty-four plugins turn security methods into agent procedures

The marketplace manifest at commit 123037e lists 44 plugins. Most target work that benefits from a fixed sequence: map a codebase before hunting bugs, inspect state-changing contract entry points, verify a suspected false positive, compare code with a specification, or search for variants of a known flaw. Others wrap concrete tools such as Semgrep, CodeQL, YARA, Burp project parsing, SARIF, fuzzers, and compiler output. The catalog reads like a security team's internal playbook made installable.

That specificity is the appeal. A general coding assistant can ask sensible security questions, but these plugins tell it which artifacts to gather, which shortcuts to reject, and where to stop for review. The repository's authoring guide requires security skills to name dangerous rationalizations and match prescriptiveness to risk. None of that guarantees a correct finding. It gives the agent a repeatable route and gives a reviewer a better record of how it reached the result.

One marketplace serves Claude Code, Codex, and ChatGPT workspaces

Claude Code installs the repository as a plugin marketplace, then lets the user choose plugins from its menu. Codex reads the same Claude marketplace metadata, so the maintainers explicitly reject duplicate Codex sidecar files that could drift. ChatGPT workspace imports use .claude-plugin/marketplace.json; skills with agents/openai.yaml also need display-name and short-description fields. commit 123037e, pushed September 16, 2026, fixed that interface metadata and tightened the corresponding validation.

The support statement has edges. The root README documents those 3 destinations, while issue 8 asks about Cursor compatibility and issue 12 asks for Gemini CLI instructions. Many plugins invoke host tools, hooks, or namespaced subagents, so copying a SKILL.md into another client does not prove the complete plugin works there. If your client is absent from the installation section, test component discovery and every tool call before trusting an audit workflow.

What happened when we ran it

Our sandbox identified plugins/constant-time-analysis/ as the runnable Python project. Installing it took 19 seconds, added 36 packages, and used 37 MB on disk. The wheel build succeeded in 1 second. Pip-audit reported 0 known vulnerabilities in that installed environment. These figures describe one plugin at commit 123037e, not the dependencies or runtime cost of all 44 marketplace entries.

The scoped project exposed no test script or target, so our harness skipped tests. It would be wrong to convert that result into a pass or a failure. The full repository documents a separate make check route, but that was outside the supplied lab run. Our checkout contained 1,441 files, about 92,367 lines of source, and occupied 8.7 MB. The repository had 3 CI workflow files, no Dockerfile, and no repository-level tests directory in the harness signal.

Constant-time analysis covers 7 architectures and many runtimes

The measured plugin includes ct-analyzer, a Python 3.10+ command that inspects compiled assembly or runtime bytecode for operations whose timing may depend on secrets. Its README names 7 architecture targets and compiler paths for GCC, Clang, Go, Rust, and Swift. It also documents bytecode inspection for JVM languages, C#, PHP, JavaScript, TypeScript, Python, and Ruby. JSON and GitHub annotation output make the command usable in CI.

Results still need interpretation. Integer division and selected floating-point instructions are errors, while conditional branches are warnings because their risk depends on whether the condition carries secret data. The Claude plugin adds language references and triage guidance around the command. System requirements vary by target: TypeScript needs a compiler, PHP's preferred path uses VLD, and compiled languages need their compiler or cross-compiler. The 37 MB Python install does not include that whole external toolchain.

The full check needs 5 host tools and still omits CI-only work

The contribution guide says make check needs uv, shellcheck, shfmt, bats, and Node on the path. It runs metadata validators, formatting and shell checks, plugin Python suites, JavaScript tests, and validator self-tests. Two loadability checks stay in CI because they require Claude Code and Codex. Review-walkthrough browser checks need Chrome or Chromium, and a version-increment rule needs a pull-request base reference. A local green check is useful evidence with stated omissions.

Open reports show why plugin-level testing matters. Issue 298 says the static-analysis skill silently omits files larger than 1 MB under Semgrep's default, while issue 299 identifies a recommended ruleset URL that returns 404. Issue 289 reports that CRLF conversion breaks packaged workflows on Windows and that an fp-check stop hook can deadlock a session. These reports concern named plugins and conditions; they do not establish that every plugin shares the fault.

September activity matters more than the missing release page

GitHub showed 7,127 stars, 26 combined open issues and pull requests, and a last push on September 16, 2026. Pull request 311 was active on September 17. The latest-release endpoint returned no release, but current commits and issue activity show that work is continuing. Plugin versions live in the marketplace and individual manifests, and the contribution rules require a version bump when a plugin changes so clients can receive it.

The repository uses CC BY-SA 4.0, a meaningful difference from permissively licensed code collections. Review that obligation before adapting and distributing the skill text inside a private marketplace. For a security team already able to judge scanner output, the 44-plugin catalog is worth selective adoption. For a developer hoping an agent can approve a system without expert review, the constant-time plugin's split between errors and warnings explains why that expectation is unsafe.

Alternatives

ProjectWhat it isPick it when
Anthropic Skills gh↗Anthropic's public collection of Agent Skills and worked examples.pick this instead when official format examples matter more than a security-specialist catalog.
Sentry SkillsAgent Skills used by Sentry for its own software-development work.pick this instead when application development and Sentry's engineering practices are closer to your daily work.
Superpowers gh↗A general agent development method built around planning, testing, and review.pick this instead when you need a broad coding workflow rather than specialist security procedures.

What people are saying

  1. [github-trending] trailofbits/skills
  2. [github-trending] google/skills
  3. [github-trending] Jeffallan/claude-skills
  4. [github-trending] plugin87/ux-ui-agent-skills
  5. [github-trending] tech-leads-club/agent-skills
  6. [github-trending] google-gemini/gemini-skills

Sources

  1. Trail of Bits Skills repository and marketplace README
  2. Constant-Time Analyzer plugin README
  3. Trail of Bits skill authoring and validation guide
  4. Windows workflow packaging report
  5. Static analysis file-size omission report
  6. Static analysis ruleset link report

More ai tools reviews

rowboat · superpowers-zh · yolov5 · eve · MemOS · LongCat-Video · the whole board →