shap review

shap review

SHAP is a Python toolkit for explaining how model inputs contribute to individual predictions and broader model behavior. It gives data scientists a common set of explainers and plots for tree models, neural networks, text pipelines, and functions that do not expose their internals.

shap/shap · ★ 25,727 · Jupyter Notebook · v0.52.0 · reviewed by Keyoor

We ran it

Cloned commit 7303655 on 05 Sept 2026, run in a fresh sandbox (3 CPUs, 8 GB, Debian container, python toolchain). How we test →

Lab card: what happened when we ran shap

Screenshot of shap (shap.readthedocs.io)

Install	✓ · 91s	48 packages · 284 MB
Build	✓ · 4s
Tests	✗ · 5s	ran, no count parsed
Known vulns	0	(pip-audit)
Repo	603 files	~91,291 lines of source · 408 MB · 9 CI workflows · tests dir

Answers from our run

Does shap build from source?

Dependencies installed in 91 seconds (48 packages), and the build succeeded in 4 seconds. We cloned commit 7303655 into a clean Debian container with 3 CPUs and no project-specific setup.

Do shap's tests pass?

The test command failed in our container, and its output did not report a pass or fail count.

Does shap have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use shap?

Teams seeking causal conclusions from observational features: the README defines SHAP as an explanation of model output and credit allocation, not a causal-inference package.

What are the alternatives to shap?

InterpretML, Captum, LIME. Our SHAP install took 91 seconds and its 4-second build passed, but test collection stopped in 5 seconds because matplotlib was missing.

Who it’s for

Data scientists who need feature-attribution checks for models already in development or production.

ML engineers working with XGBoost, LightGBM, CatBoost, scikit-learn, PySpark, TensorFlow, PyTorch, or transformer pipelines.

Reviewers who want per-prediction explanations plus dataset-level plots from the same result object.

Researchers prepared to choose a suitable explainer and background dataset for each model.

Who it’s NOT for

Teams seeking causal conclusions from observational features: the README defines SHAP as an explanation of model output and credit allocation, not a causal-inference package.

Sparse text pipelines that cannot densify their matrices: open issue 4256 reproduces a TreeExplainer failure on a CSR bag-of-words matrix.

Users expecting GPU Tree SHAP from the ordinary package install: the README requires a source build, a CUDA toolkit, and SHAP_ENABLE_CUDA=1.

Projects pinned to old Scientific Python dependencies: SHAP follows SPEC 0 minimums and says bugs on older versions may not be fixed.

Teams requiring a settled 1.0 API: the open 1.0 tracker still lists explainer calls, plots, DeepExplainer compatibility, GPU distribution, and cleanup work.

Setup reality

Our sandbox install succeeded in 91 seconds, adding 48 packages and using 284 MB. The build passed in 4 seconds. Tests stopped after 5 seconds with exit code 4 because tests/conftest.py imported matplotlib.pyplot, but matplotlib was unavailable. Pip-audit found 0 known vulnerabilities.

Basic installation uses PyPI or conda-forge and needs no credential or external service. GPU Tree SHAP is a different path: the README calls for a source install, an available CUDA toolkit, and the SHAP_ENABLE_CUDA=1 environment setting. Model-specific examples also require their own frameworks and data.

The checked-out repository was 408 MB across 603 files and about 91,291 source lines. SHAP follows Scientific Python SPEC 0 for minimum dependency versions. The failed collection step means our run did not reach any test cases, so it says nothing about whether the explainers themselves would have passed.

v0.52.0 spans tree models, text pipelines, and neural networks

SHAP turns a model prediction into feature contributions relative to a baseline. Version 0.52.0 documents fast native paths for XGBoost, LightGBM, CatBoost, scikit-learn, and PySpark tree models, plus explainers for TensorFlow, PyTorch, transformer pipelines, and arbitrary prediction functions. The same result can feed waterfall, force, scatter, beeswarm, bar, text, and image views. That breadth is the main reason to choose SHAP: one vocabulary can follow a model from a notebook investigation into a review report.

The common interface can hide meaningful differences. TreeExplainer has exact algorithms for supported tree ensembles. DeepExplainer and GradientExplainer make different approximations for neural networks. KernelExplainer is model agnostic but the README says it is slower than model-specific methods. A responsible implementation chooses the explainer, background distribution, output, and evaluation budget on purpose. Calling shap.Explainer(model) is convenient, yet the resulting chart still inherits every choice made around that call.

The 603-file checkout is larger than the one-line install suggests

Our commit 7303655 checkout contained 603 files, about 91,291 source lines, and occupied 408 MB before installation. The README makes the normal path look simple because it is: pip install shap or the conda-forge equivalent. That command is enough for many CPU examples, with no API account or hosted service. The surrounding model remains your responsibility, so an XGBoost example needs XGBoost and a transformer example needs its model framework and weights.

GPU Tree SHAP is a separate build decision. The README requires a source installation with the CUDA toolkit present and SHAP_ENABLE_CUDA=1 set. Release v0.52.0 also moved the native bindings to nanobind and the build system to scikit-build-core plus CMake. Teams maintaining internal wheels should test that toolchain on every supported platform instead of assuming the ordinary PyPI path proves the CUDA extension will compile.

What happened when we ran it

Our sandbox installed 48 packages in 91 seconds and used 284 MB on disk. The build succeeded in 4 seconds. Pip-audit reported 0 known vulnerabilities in the installed dependency set. Those are encouraging repository checks for commit 7303655, though they do not measure explanation speed, output quality, or compatibility with any particular trained model. SHAP performance depends heavily on the chosen explainer, model shape, background data, and number of samples, none of which this run benchmarked.

Pytest failed with exit code 4 after 5 seconds, before it collected tests. The log shows tests/conftest.py importing matplotlib.pyplot, followed by ModuleNotFoundError: No module named 'matplotlib'. We did not infer why the package was absent, and 0 test results would not be a fair substitute for a suite outcome. The useful finding is exact: the checked-out commit's test command could not begin its suite in our stated fresh Debian environment.

Sparse TreeExplainer inputs still have a reported failure

GitHub listed 981 combined open issues and pull requests when we fetched the project. Open issue 4256 includes a short reproduction where a scikit-learn random forest accepts a CSR bag-of-words matrix but TreeExplainer fails while checking missing values. Converting the matrix to a dense array avoids the reported error, but the reporter says the real feature matrix is too large for that workaround. Text teams using sparse inputs should reproduce this case before adopting SHAP as a required audit step.

Interpretation also needs restraint. SHAP explains how a model's output changes relative to a chosen expectation; it does not establish that changing a feature in the world causes the predicted outcome to change. Correlated inputs can make the background assumption especially consequential. For regulated or high-stakes work, keep the model version, explainer class, masker or background dataset, output scale, and library version with every exported explanation. A picture without that context is difficult to reproduce and easy to overread.

September 2026 activity is current, while 1.0 remains unfinished

The last push was September 3, 2026, and recently updated issues and pull requests had activity on September 4. GitHub reported 25,727 stars and 981 combined open issues and PRs. The latest release, v0.52.0, was published May 28, 2026 with the native-build migration, dependency-minimum changes, GPU parity fixes, plot corrections, and added tests. That mix shows active maintenance; the large open count should be read as both user interest and triage load, not as 981 confirmed defects.

An open 1.0 tracker updated September 2 lists unfinished work around the modern explainer call, documentation, plot consistency, DeepExplainer compatibility, GPU distribution, and old pull requests. SHAP is mature enough for serious use, but its maintainers have not declared the interface finished. Pin the version, test the exact model path, and review attribution assumptions with the people who own the model. The 91-second install is the easy part; deciding whether the explanation answers the right question takes longer.

Alternatives

Project	What it is	Pick it when
InterpretML	A Python package combining interpretable models with several black-box explanation methods.	pick this instead when glass-box models and a dashboard-style explanation workflow matter as much as post-hoc attribution.
Captum	A PyTorch-focused model interpretability library with attribution methods for tensors and layers.	pick this instead when the whole model stack is PyTorch and framework-specific hooks are more useful than cross-framework coverage.
LIME	A focused package for local surrogate explanations of classifiers and regressors.	pick this instead when you want a smaller local-explanation method and do not need SHAP's explainer and plotting range.