mrkeyoor.comAsk

janestreet/agent_skills

agent_skills review

Use Jane Street's agent_skills if you want an ongoing lesson built around questions, a skill map and repeated checks, and will maintain your own adaptations.

Not run yet

AI Toolsevaluationupdated 20 Nov 2026
+127 / 1dstars / 7d

Jane Street's agent_skills is a collection of instructions for AI agents and chatbots. Its socratic-tutor skill describes an ongoing lesson with diagnostic questions, a skill map, spaced review and a final assessment.

Verdict

Use Jane Street's agent_skills if you want an ongoing lesson built around questions, a skill map and repeated checks, and will maintain your own adaptations. Skip it if you want one-off explanations or upstream acceptance of rewritten teaching text. Our attempt at commit 469868f stopped at harness prerequisites. For the answer suggestions reported in Claude Code issue #16, disabling Prompt suggestions in /config worked for the reporter.

Infographic on how agent_skills works, its uses, what we have not run yet, common pitfalls and who should use it
We have not run agent_skills yet; the middle band lists what our review has not tested.

We ran it

Screenshot of agent_skills (github.com/janestreet/agent_skills)
github.com/janestreet/agent_skills

Answers from our run

Did you run agent_skills yourself?

No. GitHub reports no primary language for it, and it carries no manifest our lab installs from, and no Dockerfile, so there was nothing standard to install, build or test. This review is written from the repository's own documentation.

Who should not use agent_skills?

People seeking a one-off explanation: the tutor instructions tell the assistant to answer that normally rather than start a lesson.

What are the alternatives to agent_skills?

Socratic Discovery, socrates-skill. Use Jane Street's agent_skills if you want an ongoing lesson built around questions, a skill map and repeated checks, and will maintain your own adaptations.

SetupN/ANeither documented usage route was reproduced in a compatible agent or chatbot; the container-oriented harness stopped at prerequisites.
Docs4/5There is nothing to install; the README is a navigable 1,536-character document with 5 headings.Editor’s note: The README and pinned instructions describe usage and teaching controls, but their completeness during an actual lesson was not assessed.
Community5/5Last push on 2026-11-17, 10,370 stars and maintainers answered or closed 9 of the 9 most-discussed issues we sampled.Editor’s note: Issue #16 contains a reporter-confirmed workaround, but sampled discussions do not measure support capacity or broad learner adoption.
Maturity1/5No tagged GitHub release and repository created 2026-11-03.Editor’s note: Our lab completed no tutoring session or learning comparison, leaving grading reliability and lesson outcomes unassessed.

How we score

Discussed on

  1. hnSocratic Tutor723 points
  2. hnAgent Skills from Jane Street4 points

Who it’s for

Learners who want to answer questions and explain their reasoning.
Developers interested in questioning themselves about concepts behind their LLM's work, as yo_puaaa reported doing.
People willing to maintain their own adaptations of the teaching instructions.

Who it’s NOT for

People seeking a one-off explanation: the tutor instructions tell the assistant to answer that normally rather than start a lesson.
Contributors seeking upstream acceptance of rewritten skill text: Jane Street asks people to keep adapted copies themselves.
People looking for Addy Osmani's separate engineering workflow library.

Setup reality

Our November 20, 2026 sandbox recorded commit 469868f, 3 CPUs, 8g RAM and image n/a. It did not run agent_skills because it found no supported ecosystem (language null) and no Dockerfile. A separate Compose safety check failed because none of compose.yaml, compose.yml, docker-compose.yaml or docker-compose.yml existed.

The README documents copying the skill directory into a compatible agent's skills folder or pasting the skill body into a chatbot. In issue #16, a Claude Code user reported correct answer suggestions from the second or third question; disabling Prompt suggestions in /config worked for that reporter.

Jane Street's agent_skills is a collection of teaching instructions, distinct from Addy Osmani's similarly named engineering workflow library. Its current README presents socratic-tutor: a procedure for diagnosing your understanding, asking questions and revisiting earlier material. This agent_skills review follows that tutoring question rather than borrowing an engineering-library verdict. Jane Street's README identifies the project, while Addy Osmani's comparison page describes his separate library's engineering lifecycle.

Use Jane Street's tutor if you want to practise explaining your reasoning and accept a lesson organized around questions. Skip it if you want a quick explanation or an upstream home for rewritten teaching instructions. The instructions at commit 469868f distinguish ongoing tutoring from one-off questions, and the contribution policy asks people to maintain their own adapted copies. The most concrete client fix comes from a learner whose application started suggesting correct answers during questioning.

What people use agent_skills for

X user yo_puaaa reports that the tutor breaks misunderstandings into smaller questions. The learner's post describes quizzing themselves on concepts behind their LLM's work, so the attraction is active explanation rather than another generated summary. Two GitHub discussions add practical context: one concerns using the tutor inside Claude Code, and another explains why Jane Street shares these instructions in the first place.

Who What they actually did Outcome they described
X user yo_puaaa Quizzed themselves about concepts behind their LLM's work Reported smaller questions that broke down misunderstandings, like a personal tutor, in their learner report.
GitHub user sa-es-ir Used socratic-tutor in Claude Code Reported correct answer suggestions from the second or third question; confirmed the setting change worked in issue #16.
Jane Street maintainer dra27-js Shared skills the team had found useful and interesting Described sharing the team's experience rather than building a general directory in issue #2.

These accounts describe different uses of the same instructions. The learner wants to uncover a misunderstanding, the Claude Code user wants questions without answers appearing alongside them, and the maintainer wants to share an existing practice. For a prospective user, the common thread is a repeatable teaching procedure inside an assistant they already use. The repository's contribution policy follows that purpose: adaptations belong in the user's own copy.

What happened when we ran it

Our November 20, 2026 run recorded commit 469868f but stopped without running agent_skills: the sandbox found no supported ecosystem and no Dockerfile. The recorded configuration was 3 CPUs, 8g RAM and image n/a. Those are properties of our attempted run. The concrete outcome was a stopped harness, followed by a separate deployment safety failure and a successful teardown entry.

Recorded check Result What it means for you
Build/test sandbox NOT RUN: no supported ecosystem; language null; no Dockerfile The harness stopped before installation, build or tests.
Sandbox configuration 3 CPUs, 8g RAM, image n/a, commit 469868f, November 20, 2026 These values describe the recorded sandbox configuration.
Compose safety prerequisite FAILED: none of compose.yaml, compose.yml, docker-compose.yaml or docker-compose.yml existed The deployment safety step required a Compose file.
Teardown teardown: ok The cleanup entry recorded success.

The Compose safety step reported: "unsafe repo: none of compose.yaml, compose.yml, docker-compose.yaml, docker-compose.yml exists (rule: a compose file is required)". A generic run entry also says the repository was deployed with its own Docker Compose, with every port bound to 127.0.0.1. The specific failure contradicts that deployment description; the recorded sequence supports a failed prerequisite and teardown, rather than a completed deployment.

The README describes a different installation route: copying a directory or pasting instructions into a chatbot. That distinction explains the harness outcome without turning a missing Compose file into a tutoring defect. A project whose documented entry point is SKILL.md does not acquire a container-based installation requirement because a deployment harness expects one. The failed prerequisite belongs in the run record alongside the README's actual instructions.

Where people get stuck: Claude Code can reveal the answer

In issue #16, sa-es-ir reported correct answer suggestions from the second or third question. The Claude Code issue says the beginning was fine, then suggestions started supplying correct answers. jacobpenny pointed to Prompt suggestions in /config, and the reporter replied that disabling the setting worked. The reported consequence is specific: answers appeared while the user was supposed to reason through the question.

Problem Who reported or answered it Response and status
Correct answer suggestions spoil questioning sa-es-ir and jacobpenny in issue #16 Disable Prompt suggestions in /config; the reporter confirmed success. The issue is closed.
A contributor asks where to add references ArshiBansal and dra27-js in issue #2 The maintainer explained the repository's scope; the thread is closed.
An alternative's author asks about design origins Socratic Discovery's author and dra27-js in issue #6 The maintainer identified the original Socratic method as the inspiration and denied using existing agent skills as a basis.

Disabling Prompt suggestions addresses the symptom documented in #16 without rewriting the tutor. The distinction is useful because the skill tells the assistant to stop after asking a question, while the surrounding application can display a suggested response. The thread's resolution concerns that application setting. Its two comments contain both the proposed change and the original reporter's confirmation, making it the clearest practical troubleshooting result in this review.

The teaching procedure asks for demonstrated reasoning

The instructions at commit 469868f ask one question at a time and allow direct explanation after 2-3 failed attempts. The pinned teaching loop also permits short explanations for genuinely new material, then returns to questioning. That is a more precise account than saying the tutor never lectures: questioning is the main procedure, with explicit exceptions for explanation.

The session begins by dividing a topic into 4-10 concrete subskills, with fewer for narrow topics. It then calls for 3-5 diagnostic questions before teaching begins. The assistant shows an initial map and proposes a starting point, which the learner can override. These instructions give the lesson a visible structure: a learner can see the current subskill and ask to change difficulty rather than relying on a conversation's natural drift.

The map uses levels 0-5, from unaware to being able to teach a concept and handle edge cases. Raising a level requires demonstrated performance across at least two independent checks, including unprompted recall. The procedure also calls for revisiting earlier subskills and lowering a level after a failed review. At completion, it specifies a mixed assessment with at least one question per subskill and an integrative problem.

How to start a lesson and carry it forward

The README gives two usage routes: copy the skill directory into a compatible coding agent or paste its body into a chatbot. For the first route, Jane Street's usage instructions say to place the directory in the agent's skills folder and ask it to use the skill, with "teach me X" as the example. The second route uses the same teaching text in a chat or custom/system prompt.

The pinned instructions make the intended request explicit through phrases such as "teach me", "quiz me" and "drill me". A one-off "explain X" request should receive an ordinary answer instead of triggering the diagnostic. During a lesson, "show progress" displays the map, "harder" and "easier" adjust difficulty, and "just tell me" overrides Socratic mode for the current question. These are documented conversational controls, rather than shell commands or a separate application API.

For continuity, "save state" asks the assistant to emit a compact block containing the topic, levels, misconceptions and covered material. "resume from:" accepts that block in another session. The command definitions at 469868f describe copying conversation state between sessions. Keeping that distinction clear helps a learner understand what to save: the emitted block itself carries the lesson's recorded context.

Contributions stay close to Jane Street's own practice

Jane Street asks users to keep adapted skill text in their own copies. Its contribution policy says the skills are exported from an internal repository and that maintainers are not seeking new skills or pull requests changing existing text. It welcomes issues when behavior differs from the description. That policy matters before investing time in a teaching-style rewrite intended for upstream acceptance.

The same boundary appears in issue #2, where a requester asks for references to existing projects and another contributor asks where to make changes. dra27-js responds that the repository shares what the team has found useful and interesting rather than serving as a general agent-skills resource. For someone adapting the tutor, maintaining a personal copy follows the published policy; reporting a mismatch between instructions and behavior follows the issue guidance.

agent_skills vs Socratic Discovery and socrates-skill

Socratic Discovery's author describes open-ended discovery, while Jane Street's README describes a skill map, spaced review and assessment. In issue #6, the alternative's author names exploration modes, pacing signals and reflection on thinking patterns. That attribution gives readers a concrete teaching-style distinction without turning the discussion into a ranking of learning results.

Choice Description in the linked source When to consider it
Jane Street agent_skills The README describes diagnostic placement, levels 0-5, spaced review and final assessment. You want a lesson organized around questions and demonstrated understanding.
SevenTianyu/socratic-discovery-skills Its author describes open-ended discovery, exploration modes and pacing signals in issue #6. You prefer the author's stated emphasis on exploration and reflection.
bevibing/socrates-skill A requester names it as an existing project worth distinguishing in issue #2. You want another named Socratic teaching project to inspect.

The decision supported by these descriptions concerns the lesson you want. Jane Street spells out placement, repeated checks and completion conditions. Socratic Discovery's author emphasizes exploration and pacing. The reference to socrates-skill gives a third project name rather than a detailed feature description. Addy Osmani's library belongs to a separate decision about engineering workflows, which is why its lifecycle comparison does not determine the choice of tutor here.

Outdated claims, corrected

The tutor permits direct explanations, and Anthropic's announcement identifies Agent Skills as its open standard. These correct two promotional claims: samurai1269's X post says the tutor "won't lecture", while the BuildRun issue credits Jane Street with standardizing modular behavior through SKILL.md. The pinned teaching rules contain explicit explanation exceptions, and Anthropic's announcement says it published Agent Skills as an open standard.

The BuildRun promotional issue #5 describes Jane Street's tutor using the format; Jane Street's README describes sharing internally used skills. Those statements support a contributor using an existing format, rather than credit for originating the standard. Similarly, the X post's absolute promise compresses a more detailed teaching procedure into a slogan. The actual instructions distinguish questions, short introductions, repeated failed attempts and a user-requested explanation override.

The Productize page also needs its entity kept intact: its review explicitly names Addy Osmani's repository. It is not a Jane Street tutor review. Identifying the owner prevents that page's engineering feature list from becoming a claim about socratic-tutor. The correction is about which repository a statement describes, rather than treating every similarly titled page as evidence about the same software.

FAQ

Disabling Prompt suggestions in /config resolved the answer-spoiler problem reported in issue #16. The remaining questions here concern Jane Street's tutor: its design origins, contribution policy and the point at which an ongoing lesson makes sense. Each answer follows the README, the pinned teaching instructions or a named issue participant's account, keeping the choice centered on learning through questions.

How do I stop Claude Code from suggesting the answers during socratic-tutor sessions?

Turn off Prompt suggestions in /config, the setting recommended in issue #16. The reporter's confirmation says it worked after correct answers began appearing from the second or third question. This is a client-setting response to the reported symptom. The issue does not describe a patch to the tutor instructions; its resolution is the setting change suggested by jacobpenny.

How was socratic-tutor developed, and was Socratic Discovery among its references?

Maintainer dra27-js says the original Socratic method inspired the skill, rather than existing agent skills. That response in issue #6 answers Socratic Discovery's author, who asked about references and described differences between the projects. The README also says the skill was written internally and is not knowingly derived from another implementation. These are the maintainer's account of the design's origins.

Where should a contributor make changes to add references to existing skills?

The README asks people to keep adapted versions themselves and declines pull requests changing existing skill text. In issue #2, the maintainer answers the reference request by explaining the repository's purpose, rather than naming a file to edit. The current contribution policy is the concrete guidance: personal adaptations stay in personal copies, while reports of behavior differing from the description are welcome as issues.

When should I use a skill?

Use this tutor for an ongoing lesson or quiz; its intent guard excludes one-off explanation requests. The instructions at commit 469868f tell the assistant to answer "explain X" normally and ask a short clarifying question when the user's intent is ambiguous. Choose the tutor when diagnostic questions, a skill map and repeated recall checks match the learning session you want.

What we have not tested yet

Our stopped attempt produced no completed tutoring session, learning-gain measurement, retention result or model comparison. Neither README usage route, the save/resume procedure nor the Prompt suggestions workaround was reproduced by our lab. The recorded image is n/a; runtime and operating-system versions require the raw toolchain logs. The attempt also produced no task screenshots or known-answer question results, and its sandbox resource values are not tutor hardware requirements.

The positive learner report is qualitative, and the teaching-style comparison comes from an alternative's author. Controlled comparisons against ordinary prompting, Socratic Discovery, socrates-skill and human tutoring are outside this assessment. The tutor's progress levels have no demonstrated connection here to performance outside the conversation, and trading-themed promotions offer no demonstrated trading result or qualification to risk money. An engineering benchmark about other skills answers a different question from learning with this tutor.

Security review, multilingual teaching quality, reliable grading across clients and long-running lesson behavior are also outside this assessment. Rendered layout, automatic related links, recommendation modules and template content were not tested. A useful next evaluation would keep the topic, model and client constant and compare independently checked recall after ordinary explanation and after guided questioning. For now, the supported reason to choose agent_skills is its documented teaching procedure and the attributed experiences described above.

Alternatives

ProjectWhat it isPick it when
Socratic DiscoveryIts author describes open-ended discovery, exploration modes, pacing signals and reflection on thinking patterns.Consider it when that stated exploration style appeals more than Jane Street's map and assessment procedure; the comparison comes from its author in issue #6.
socrates-skillA related Socratic teaching project named by a requester in Jane Street issue #2.Inspect it when considering another named tutor; issue #2 supplies a reference request rather than a feature comparison.

What people are saying

  1. [velocity-scout] janestreet/agent_skills

Sources

  1. Jane Street agent_skills README
  2. Socratic tutor instructions at 469868f
  3. Claude Code answer suggestions, issue #16
  4. Design origins and Socratic Discovery comparison
  5. Repository scope and reference request
  6. Reported tutor use on X

More ai tools reviews