spacy review
spaCy is a Python NLP pipeline library for tokenization, sentence boundaries, part-of-speech tags, dependency parses, named entities, text classification, rule matching, and custom trained components. Its `Doc` and `Token` objects keep the original text aligned with linguistic annotations, and `nlp.pipe` batches large input sets. Models are separate Python packages, so installing spaCy alone gives you the engine and language rules, not an English tagger or entity recognizer. Current 3.8.16 follows the 3.8.15 Click dependency fix; its tag diff contains release and documentation infrastructure changes rather than a runtime API change.
spaCy is a strong fit for repeatable, token-aligned NLP pipelines where rules, trained components, and CPU batching meet. Skip it for model-specific token counts or tiny deployments, and count the separate pipeline package in every build plan.
We installed it
| Install | ✓ · 1.7s | 44 packages on disk · 253 MB |
| Import | ✓ | import spacy in 2.19s · compiled extensions · py.typed · requires Python <3.15,>=3.9 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does spacy install cleanly?
Yes. In a fresh container with an empty cache, pip install spacy finished in 2 seconds, leaving 44 packages and 253 MB on disk. pip-audit reported no known vulnerabilities.
What does spacy need to run?
Python <3.15,>=3.9, and a platform wheel with compiled extensions. In our run import spacy succeeded in 2.19s, and the package ships py.typed for type checkers.
spacy or stanza: which should you use?
stanza: Use it for Stanford neural pipelines when its language coverage and accuracy suit the task better than spaCy's speed. spaCy is a strong fit for repeatable, token-aligned NLP pipelines where rules, trained components, and CPU batching meet.
When should you not use spacy?
You only need the token IDs used by an LLM. spaCy's linguistic tokenizer does not reproduce a model vendor's BPE or SentencePiece vocabulary; use that model's tokenizer.
Use it if
- A CPU service needs token-aligned entities, tags, lemmas, sentence boundaries, or dependency trees over many documents.
- Handwritten token or phrase rules must run beside statistical components in one ordered processing pipeline.
- Training data, config files, evaluation, packaging, and inference should use the same component registry and `Doc` format.
- Batching through `nlp.pipe` and disabling unused components can reduce repeated Python overhead in a large text job.
- You only need the token IDs used by an LLM. spaCy's linguistic tokenizer does not reproduce a model vendor's BPE or SentencePiece vocabulary; use that model's tokenizer.
- A small deployment cannot absorb the environment we measured: 44 installed packages, 253 MB on disk, 47 direct dependencies, and compiled extensions before any trained pipeline is added.
- The target forbids binary wheels or local compilation. spaCy ships `.so` extensions, and source installs require a compiler plus compatible NumPy, Thinc, Blis, Cymem, and Preshed builds.
- Your task requires a transformer encoder out of the box. Core spaCy does not include one; `spacy-transformers`, a compatible model, and the PyTorch stack are separate choices.
- A hosted language model already performs a small, low-volume extraction with acceptable cost and data handling. Maintaining labels, training examples, and model packages may add more work than the deterministic pipeline saves.
Setup reality
We installed spaCy 3.8.15 in a fresh Python 3.12 Bookworm container. The install completed in 1.7 seconds, left 44 packages, and used 253 MB on disk. The package declares 47 direct dependencies, supports Python 3.9 through versions below 3.15, ships compiled .so extensions, and includes a py.typed marker. import spacy worked in 2.19 seconds. pip-audit found 0 known vulnerabilities.
That install did not include a trained language pipeline. spacy.load('en_core_web_sm') fails until the matching model package is installed. python -m spacy download en_core_web_sm selects a compatible release, but production builds should pin the model artifact and run python -m spacy validate after spaCy upgrades. The current package is 3.8.16; our size, dependency, import, and audit measurements apply to 3.8.15 only.
Wheels avoid compilation on supported platforms. A missing wheel turns installation into a native build with compiler and Python headers. GPU processing needs a matching CuPy extra; transformer pipelines also add spacy-transformers, PyTorch, and a compatible trained pipeline. Blank language objects supply tokenization rules but no statistical tagger, parser, lemmatizer, or named entity component unless you configure or train them.
Load the pipeline once per process. Use nlp.pipe for batches and disable components whose annotations are not consumed. Multiple worker processes each hold model state, so higher n_process can increase memory sharply. DocBin is the supported compact format for processed documents and training examples. When upgrading spaCy or a model, validate compatibility and rerun task-level evaluation because pipeline package versions and runtime versions are coupled.
Patterns
Load a model and read named entities load-model-extract-entities
import spacy
nlp = spacy.load('en_core_web_sm')
doc = nlp('Apple opened a store in Mumbai in August.')
for ent in doc.ents:
print(ent.text, ent.label_, ent.start_char, ent.end_char)Install a compatible `en_core_web_sm` package first. spaCy itself does not contain this entity model.
Tokenize with a blank language object tokenize-without-model
import spacy
nlp = spacy.blank('en')
doc = nlp.make_doc('Ship order #A-42 today.')
print([(token.text, token.idx) for token in doc])A blank pipeline has language tokenization rules but no trained tagger, parser, or entity recognizer.
Inspect linguistic token fields read-token-annotations
doc = nlp('The courier delivered two boxes.')
for token in doc:
print(token.text, token.lemma_, token.pos_, token.dep_, token.head.text)These fields need the matching trained components. Attributes with a trailing `_` return readable strings instead of integer IDs.
Process a stream in batches batch-process-texts
texts = load_texts()
for doc in nlp.pipe(texts, batch_size=128):
save_entities([(ent.text, ent.label_) for ent in doc.ents])Start with one process and measure. Each additional worker can load its own model state and raise memory use.
Load only the components a task needs disable-unused-components
nlp = spacy.load(
'en_core_web_sm',
disable=['parser', 'lemmatizer'],
)
for doc in nlp.pipe(texts):
save_entities(doc.ents)Disabling a component also removes the annotations it creates. Check downstream custom components before changing the order or enabled set.
Add lightweight rule-based sentence boundaries split-sentences
nlp = spacy.blank('en')
nlp.add_pipe('sentencizer')
doc = nlp('First shipment arrived. The second is late!')
for sentence in doc.sents:
print(sentence.text)The sentencizer uses punctuation rules. Use a parser or trained sentence recognizer when punctuation alone is insufficient.
Match a token sequence match-token-pattern
from spacy.matcher import Matcher
matcher = Matcher(nlp.vocab)
matcher.add('ORDER_ID', [[
{'ORTH': '#'},
{'IS_ALPHA': True},
{'ORTH': '-'},
{'IS_DIGIT': True},
]])
for match_id, start, end in matcher(doc):
print(doc[start:end].text)Each pattern dictionary consumes one token. Inspect the tokenizer output when punctuation does not match the expected sequence.
Match a case-insensitive terminology list match-phrase-list
from spacy.matcher import PhraseMatcher
matcher = PhraseMatcher(nlp.vocab, attr='LOWER')
terms = ['heat pump', 'solar inverter', 'battery storage']
matcher.add('PRODUCT', [nlp.make_doc(term) for term in terms])
for _, start, end in matcher(doc):
print(doc[start:end].text)Build phrase patterns with `nlp.make_doc` so the full processing pipeline does not run over the terminology list.
Insert known entities before the model add-entity-rules
ruler = nlp.add_pipe('entity_ruler', before='ner')
ruler.add_patterns([
{'label': 'PRODUCT', 'pattern': 'TonerJet 4200'},
{'label': 'ORG', 'pattern': [{'LOWER': 'acme'}, {'LOWER': 'logistics'}]},
])
doc = nlp('Acme Logistics ordered a TonerJet 4200.')Pipeline position controls overlap behavior. Test how rules and the statistical NER resolve the same span.
Register a serializable pipeline component register-custom-component
from spacy.language import Language
from spacy.tokens import Doc
Doc.set_extension('order_ids', default=[])
@Language.component('order_id_collector')
def order_id_collector(doc):
doc._.order_ids = [span.text for span in find_order_ids(doc)]
return doc
nlp.add_pipe('order_id_collector', last=True)A component must return the Doc. Register extensions once during startup and give the component a stable factory name for saved pipelines.
Write processed documents with DocBin serialize-docs
from spacy.tokens import DocBin
doc_bin = DocBin(store_user_data=True)
for doc in nlp.pipe(texts):
doc_bin.add(doc)
doc_bin.to_disk('corpus.spacy')
loaded = DocBin().from_disk('corpus.spacy')
for doc in loaded.get_docs(nlp.vocab):
consume(doc)Use the same vocabulary and compatible pipeline definitions when restoring integer-backed strings and custom data.
Check installed model compatibility validate-model-packages
python -m spacy validateRun this after changing spaCy versions. It reports installed pipeline packages that do not match the runtime compatibility table.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| stanza | PyPI | Use it for Stanford neural pipelines when its language coverage and accuracy suit the task better than spaCy's speed. |
| transformers | PyPI | Use it when the job starts with a pretrained transformer architecture and you can provide the model, tokenizer, and compute. |
| nltk | PyPI | Use it for teaching, corpus access, and classic NLP algorithms where an application pipeline and trained production model are unnecessary. |
More ai / ml guides
openai · mcp · huggingface-hub · scikit-learn · tiktoken · @modelcontextprotocol/sdk · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.