Our run installed neuron-explorer in 15 seconds and built it in 1 second at commit 7ec363d. Use it to investigate Qwen3.5-4B interventions through saved responses and controls. Readers who need established neuron meanings or interchangeable model support should choose a different workflow. The useful distinction is between inspecting evidence of an intervention and deciding what that intervention explains: the interface gives you several ways to examine the former.
What people use neuron-explorer for
lyramakesmusic/neuron-explorer explores and steers Qwen3.5-4B neurons. The upstream README describes pushing individual MLP neurons up and down on several prompts, keeping every rollout and arranging the results over a 3D model view. Its list ranks units by word-cluster mass against random-direction controls. That gives a reader a concrete comparison to inspect: the selected intervention, the unsteered response and responses produced by control directions.
The supplied reports describe cached browsing, adaptations to other model families and related tools built from the project. These are different uses of the same starting point. Cached browsing helps someone read existing experiment results; adapting the code makes model selection part of the research task. The following accounts identify who reported each use, so a community observation stays distinct from our source installation and build.
| Who used it | What they did | Reported outcome |
|---|---|---|
| CleanFix_Dev's cached browsing report | Browsed cached data in no-model mode on Windows | Map and neuron details worked; one attention-head view returned an error |
| sambhavdixitpro's model adaptation report | Modified the explorer for Qwen3.5-0.8B, Gemma-3-1B and DeepSeek-Coder-1.3B | Reported neuron hunting across three model families |
| NapoleonPK's Gemma port | Adapted the explorer to Gemma | Reported checks against real E4B and E2B checkpoint headers |
| whp199's issue #12 comment | Borrowed code for theodolite | Reported building a related tool for Gemma4-31b-it |
CleanFix_Dev's account gives a useful first task: open the cached map, inspect a neuron and try the attention-head view that returned an error. The other accounts show why someone might take on a larger adaptation. For example, sambhavdixitpro explicitly names three smaller models, while whp199 names a separate Gemma tool. Choose the upstream project for its documented Qwen workflow; choose an adaptation because its model target matches the question you want to investigate.
What happened when we ran it
Our source sandbox run at commit 7ec363d on 2026-11-17 installed in 15 seconds and built in 1 second. The recorded configuration was 3 CPUs, 8 GB RAM and image label ghcr.io/astral-sh/uv:python3.12-bookworm. The installation recorded 35 dependencies using 39 MB on disk. These figures describe the source-run harness, whose installation, build, test and audit outcomes are separated below from the earlier registry recipe.
| Source-run observation | Recorded result | Verdict | Why |
|---|---|---|---|
| Install | Succeeded in 15 seconds | Completed | The recorded dependency step exited successfully |
| Dependencies | 35 recorded; 39 MB on disk | Recorded footprint | These figures belong to this installation |
| Build | Succeeded in 1 second | Completed | The configured build step exited successfully |
| Tests | Skipped: no tests script or target | Skipped | The harness found no test target |
| pip-audit | 0 known vulnerabilities | No known advisories reported | This is the recorded dependency-audit result |
| Repository scan | 52 files, approximately 8,576 source lines, 211.5 MB checked out | Recorded checkout footprint | Source checkout size is separate from dependency disk use |
| Automation scan | 0 CI workflow files, no Dockerfile, no tests directory | Those files absent from the scan | These are repository-scan observations |
The separate earlier hands-on run record names commit 7ec363d2d968 and a configuration of 2 CPUs, 4 GB RAM, no published ports and no secrets. Its quickstart-lib recipe cloned in 14 seconds, then attempted uv pip install neuron-explorer in a fresh environment. The installation step failed in 1 second, exiting with code 1. Its log identifies Python 3.12.12 and says the package was not found in the registry.
The install-failed finding records unsatisfiable requirements, and teardown returned ok. Although the earlier summary describes a PyPI installation, its step log records failure. The outcome here follows that step log. The successful source harness and the failed registry recipe are separate observations with different configurations; the README's quickstart starts inside the repository with uv sync and uv run python explorer/server.py.
Where people get stuck: choose the right execution mode
The README separates cached experiment data, local Qwen3.5-4B loading and CUDA 13 sweeps. Start by deciding which output you need: an existing saved response, a response to your own text or a new collection of interventions. The repository ships caches under explorer/cache/, reads sweep runs from runs/ and reads per-layer statistics from stats/. Those locations make the distinction concrete before you choose commands or hardware.
| Symptom or decision | Evidence | Practical response |
|---|---|---|
| Package name cannot be resolved | Our earlier registry attempt stopped at installation | Use the documented repository quickstart |
| Cached attention-head page errors | CleanFix_Dev reports an error alongside working map and neuron-detail views | Include that view in an initial interface check |
| Need fresh live responses | README documents local loading through MPS, CUDA or CPU | Plan for loading Qwen3.5-4B locally |
| Need fresh sweep data | README specifies CUDA 13 and a steering fork of vLLM | Follow the separate sweep setup |
| Want a Gemma neuron with the same ID | The Gemma fork says Qwen neuron IDs do not carry over | Use data and identifiers for the selected model |
The upstream sweep instructions specify RhizoNymph's vLLM fork on a CUDA 13 GPU pod. The explorer's local-loading description separately names MPS, CUDA and CPU. Read those as instructions for different jobs. If your immediate aim is to inspect a saved intervention, begin with the cached material. If your aim is to generate another sweep, the setup includes the sweep environment and the files it produces.
A neuron label is a hypothesis you can investigate
The README pairs neuron names with steered rollouts, unsteered baselines and random-direction controls. Compare those outputs when investigating a displayed label: read the generated text, check the baseline, then examine the controls and other prompts. The documented views also include activation filtering, token lenses and connections. This reading sequence is an editorial recommendation grounded in those features, keeping interpretation attached to observable text and interventions rather than letting a neuron's name supply the explanation.
Issue #13's scoring proposal asks whether word-count metrics miss changes in structure, format or reasoning style. Its author connects that question to the apparent mid-layer dip and proposes embedding displacement plus consistency across prompts as another scoring approach. The issue says the rollout embeddings already exist and describes the remaining scoring as CPU work. It presents an experiment to compare different measures of intervention effects.
The proposal gives a reader a more specific question than whether a label sounds plausible: does a neuron's intervention move its rollouts away from random controls across prompts and doses? The existing list supplies word-cluster rankings, while the proposed analysis uses embedding displacement and consistency. Reading the outputs alongside their scoring method helps distinguish what the ranking counts from what a researcher wants to learn about the generated text.
Fresh sweeps create more evidence to manage
The README says sweep outputs retain text, token IDs, logprobs and top-k data, with optional activation captures. It also lists separate scripts for ranking, embeddings, layout and connections. A fresh sweep therefore produces material that must be organized before it becomes the map a reader sees. The 39 MB dependency footprint from our installation describes a different quantity from the files created by this experiment workflow.
Choose a specific comparison before launching a broad sweep: which prompt, which intervention and which baseline will make the result interpretable? Preserve the corresponding outputs so another reader can inspect the same evidence. The README's documented unsteered baselines and random-direction controls provide useful anchors for that decision. Starting with a saved experiment also lets you learn how the resulting text, rankings and views fit together before creating another collection.
Claude can read files and run a shell
The README says the Claude integration has file tools and a shell in your home folder. It requires an installed, logged-in claude CLI and uses the explorer as an MCP server. Its documented tools read neurons, cones and lenses, run model operations and move interface views. For an operator deciding whether to enable it, the file and shell access belongs in the same setup decision as the model tools.
The README also says the server answers pages and tools only on the local machine. Keep that documented local arrangement when following its setup instructions, and choose the working account with the agent's file access in mind. The interface supports English and Simplified Chinese: it follows the browser language on the first visit, provides a header menu for switching languages and retains the selection in that browser. Both details come from the upstream documentation.
neuron-explorer vs the Gemma fork and Koen Korevaar's explorer
The Gemma fork uses model-specific data, while Koen Korevaar's explorer presents prepared Qwen3-0.6B examples. The upstream project documents Qwen3.5-4B interventions and local inference over your own text. Choose according to whether you want to study existing examples, operate the Qwen workflow or build experiments around Gemma. The comparison is about the documented task each project supports and the work its author asks you to undertake.
| Project | Evidence and scope | Pick it when |
|---|---|---|
| lyramakesmusic/neuron-explorer | README documents Qwen3.5-4B, shipped caches, local loading and sweep tools | You want the original Qwen intervention workflow |
| NapoleonPK/neuron-explorer-gemma | Author reports checkpoint-header checks; Qwen caches were removed because their shapes do not fit Gemma | Gemma is your subject and you can generate its data |
| Koen Korevaar's Neuron Explorer | Qwen3-0.6B explorer with prepared sentence traces | You want a browser introduction using existing examples |
The Gemma fork says no caches or sweep runs ship with it. Its README explains that the Qwen caches were removed because they do not fit Gemma's shapes, so the map begins without sweep data. The fork also says Qwen neuron IDs do not carry over to Gemma. Those differences make generating model-specific data part of choosing the port; its retained Qwen findings are reference material.
Korevaar's page explicitly says visitors cannot type their own sentence and that nothing on the site runs a model. Its prepared traces, tour and glossary suit a reader learning how the views relate to one another. Someone whose question requires a new prompt should instead consider the upstream project's documented local-inference workflow. Prepared examples and an experimental server answer different reader needs, even though both use neuron maps as an entry point.
What the other reviews get wrong
OpenFree's missing-license warning conflicts with the maintainer's MIT response in issue #4. The OpenFree listing says no recognizable license was detected. In the closed license issue, the maintainer reports adding MIT so others can build on the project. The current upstream README identifies MIT and links its LICENSE file. That upstream documentation is the appropriate starting point for the code's license conditions.
The Gemma fork's README also says upstream has no license file. That description conflicts with the supplied current upstream README and maintainer response. Separately, the overview video's platform guidance should be read by execution mode: the research pack records macOS or Linux plus a dedicated GPU or Apple Silicon as its stated requirements, while CleanFix_Dev reports working cached map and neuron-detail browsing on Windows. Cached browsing is the specific counterexample to that blanket setup advice.
For setup decisions, use the upstream explorer instructions to distinguish shipped caches from local model loading, then read the separate sweep requirements. The attributed Windows account describes a particular cached-mode outcome, including an attention-head error. Keeping those details together gives a reader a useful initial check: inspect the map, open neuron details and try the attention-head view before deciding which additional workflow to undertake.
Questions people ask
The current upstream README identifies MIT as the code license, answering the reuse question raised in issue #4. The other supplied questions concern scoring, interpreting internal views and steering behavior. The answers below point to documented operations or named authors' reports, giving readers a concrete source to inspect for each question. The final section collects the testing and interpretation boundaries that apply to these answers.
Can I use and build on neuron-explorer's code, and what license applies?
The upstream README identifies MIT and links the LICENSE file. In issue #4's maintainer response, lyramakesmusic says MIT was added to make the code easier to build on. Read the linked license for its conditions when planning reuse. This answers the question in that issue using the maintainer's response and current documentation rather than the older directory warning.
Is the mid-layer steering dip real, or does word-count scoring miss changes in structure, format or reasoning style?
Issue #13 proposes embedding displacement and cross-prompt consistency to investigate that question. Its author says existing rollout embeddings make the scoring work a CPU task. The issue frames the analysis as a way to distinguish a word-count blind spot from an effect also visible at the embedding level. The scoring proposal describes the comparison to make and the existing files it would use.
Can you visually trace a thought through a language model's layers?
The README documents activation filtering, token lenses, neuron connections and comparisons of steered responses. Those are concrete operations a reader can inspect when evaluating the broader phrase "trace a thought." Begin with a particular token, activation or intervention and compare the corresponding outputs. The upstream feature descriptions give precise names for these observations and explain how to move between the map, list and neuron views.
How can I extract control vectors and steer an LLM's behavior without retraining?
Issue #17 reports a contrastive direction reaching -49.6° open-loop in a video-steering fork. Its author also reports that adding an LQR controller did not beat saturation with the per-bin field. The issue #17 closing comment supports those attributed outcomes. For the upstream Qwen explorer, the README describes neuron pushes, saved rollouts and an unsteered baseline as starting points for examining its intervention workflow.
What we have not tested yet
Our successful source checks cover installation and build; live loading, steering quality, response speed and inference memory remain untested. Tests were skipped for lack of a target, so there is no passing-suite result. The source sandbox's image label identifies the configured image; actual runtime and operating-system versions require raw toolchain logs. We have not reproduced CleanFix_Dev's Windows account, diagnosed its attention-head error or established Windows live-inference support. That account itself covers cached mode and excludes live steering, model chat and sweeps.
Issue #13 remains open, and we have not reproduced its proposed scoring analysis. The supplied research synthesis records that embeddings cover only the first 600 characters of a rollout; effects of that boundary on scoring remain unmeasured here. A neuron's displayed label does not establish its causal meaning, and the available internal views do not establish a complete trace of a thought. The cross-model X report supplies no detailed findings, and whp199's theodolite comment supplies no validation results.
The Gemma author reports small random-model testing and checkpoint-header checks, but says real weights and the vLLM sweep have not been run. Issue #17's supplied extract does not document matched neutral and turn-left prompts or an activation-difference derivation. It reports an outcome rather than a complete reproducible control-vector procedure. These boundaries matter when choosing between inspecting prepared results, adapting the explorer and attempting a new experiment around another model.
The pack also records per-bin saturation, a white-veil effect and identity degradation in the video-steering fork; closed-loop contrastive steering and per-frame trajectories remain future work. These are attributed fork observations rather than upstream Qwen benchmarks. We have not tested either alternative side by side, exercised Claude's shell or audited server access controls. The dependency audit is not an application security assessment, and the records provide no production uptime, multi-user load or full-sweep storage measurements. External model weights and assets require their own license checks.

