sentence-transformers review
sentence-transformers 6.0.0 runs and trains retrieval models through four APIs: dense `SentenceTransformer` embeddings, `CrossEncoder` rerankers, `SparseEncoder` representations, and the new `MultiVectorEncoder` for ColBERT-style late interaction. It wraps PyTorch, Transformers, Hub downloads, pooling, similarity, evaluation, and training so application code does not have to assemble those pieces. Version 6 moves to Transformers 5, fixes half-precision scoring by calculating scores in float32, and adds separate query and document encoding for token-level multi-vector models. Our Python 3.12 import worked, but the clean environment occupied 4,863 MB before downloading a model, which rules it out for many small services.
sentence-transformers 6.0.0 consumed 4,863 MB and took 12.08 seconds to import in our sandbox, even before a model download. Install it when local training or its dense, sparse, reranking, and multi-vector APIs justify that runtime; inference-only CPU services should test FastEmbed first.
We installed it
| Install | ✓ · 26.1s | 59 packages on disk · 4863 MB |
| Import | ✓ | import sentence_transformers in 12.08s · pure Python · py.typed · requires Python >=3.10 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does sentence-transformers install cleanly?
Yes. In a fresh container with an empty cache, pip install sentence-transformers finished in 26 seconds, leaving 59 packages and 4863 MB on disk. pip-audit reported no known vulnerabilities.
What does sentence-transformers need to run?
Python >=3.10, and nothing compiled: it is pure Python. In our run import sentence_transformers succeeded in 12.08s, and the package ships py.typed for type checkers.
sentence-transformers or fastembed: which should you use?
fastembed: Use it for smaller ONNX-based embedding and reranking workers that do not need this training stack. sentence-transformers 6.0.0 consumed 4,863 MB and took 12.08 seconds to import in our sandbox, even before a model download.
When should you not use sentence-transformers?
A small CPU service cannot absorb our measured 4,863 MB base environment and a 12.08-second cold import before any model weights load.
Use it if
- You need local dense embeddings for semantic search, clustering, deduplication, or retrieval and can host the model runtime.
- A retrieval pipeline should rerank a short candidate list with a CrossEncoder after a cheaper first-stage search.
- The team wants sparse SPLADE-style or ColBERT-style multi-vector retrieval through the same training and inference project.
- You plan to fine-tune an embedding or reranker model and want the project's trainers, losses, evaluators, and Hub integration.
- A small CPU service cannot absorb our measured 4,863 MB base environment and a 12.08-second cold import before any model weights load.
- You only need a hosted embedding endpoint. A provider SDK avoids PyTorch, local model caching, device selection, and serving capacity.
- A lightweight CPU inference worker is the goal. FastEmbed's ONNX-oriented runtime is a better starting point than the default PyTorch stack.
- Your vector store accepts one fixed vector per document, but the chosen `MultiVectorEncoder` returns a variable-length token matrix. ColBERT-style indexes need different storage and scoring.
- Production cannot download from the Hugging Face Hub and nobody will bake a reviewed model revision into the image or persistent cache.
Setup reality
We installed sentence-transformers 6.0.0 in a fresh Python 3.12 Bookworm sandbox. pip completed in 26.1 seconds, left 59 packages, and consumed 4,863 MB. The distribution declares 27 direct dependencies, requires Python 3.10 or newer, is pure Python, and ships py.typed. Importing sentence_transformers took 12.08 seconds. pip-audit reported 0 known vulnerabilities in the resolved environment.
That 4,863 MB does not include a selected model download. Constructing a model by Hub ID fetches configuration, tokenizer, and weights into the Hugging Face cache. Pin a model revision, prefetch it during an image build or deployment stage, and point production at persistent cache storage. Air-gapped jobs should load a reviewed local directory and enable offline mode so a cold start cannot make an unexpected network call.
Device setup belongs before the package install on GPU machines. Install the PyTorch build matched to the available CUDA runtime, then verify the requested device with a real encode call. Version 6 requires Transformers 5 and PyTorch 2.2 or newer. Training, ONNX, OpenVINO, image, audio, and video support use separate extras; install only the path you test, because each adds another dependency surface.
Dense and sparse encoders return batch-shaped representations, while MultiVectorEncoder returns one 2D tensor per input because token counts differ. Use encode_query and encode_document for those asymmetric models. MaxSim scoring can allocate a large query-token by document-token intermediate; lower chunk_elements when memory is tight. CrossEncoders score every query-passage pair, so rerank a shortlist rather than an entire corpus.
Patterns
Create dense sentence vectors encode-dense-text
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode(["cats purr", "dogs bark"])
print(embeddings.shape)The constructor downloads files on first use unless the model already exists in the Hub cache or at a local path.
Use asymmetric retrieval prompts encode-query-document
queries = model.encode_query(["how do cats communicate?"])
documents = model.encode_document(["Cats often communicate by purring."])
scores = model.similarity(queries, documents)Query and document methods preserve model-specific prompts. Plain `encode` can produce the wrong representation for an asymmetric checkpoint.
Prepare vectors for dot-product search normalize-for-dot-product
vectors = model.encode(
documents,
normalize_embeddings=True,
convert_to_numpy=True,
)Unit-normalized vectors make dot product equivalent to cosine similarity. Match this choice to the vector index metric.
Search a small in-memory corpus semantic-search
from sentence_transformers import util
corpus_vectors = model.encode(corpus, convert_to_tensor=True)
query_vector = model.encode("Berlin population", convert_to_tensor=True)
hits = util.semantic_search(query_vector, corpus_vectors, top_k=5)[0]This helper compares in memory. Move a large or shared corpus into an index that supports the chosen similarity metric.
Rerank first-stage candidates rerank-passages
from sentence_transformers import CrossEncoder
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
ranked = reranker.rank("How many people live in Berlin?", passages, return_documents=True)A CrossEncoder runs the model on every query-passage pair. Feed it a shortlist from the first retrieval stage.
Generate sparse SPLADE representations encode-sparse-text
from sentence_transformers import SparseEncoder
encoder = SparseEncoder("naver/splade-cocondenser-ensembledistil")
sparse = encoder.encode(["error handling in Python"])
stats = SparseEncoder.sparsity(sparse)The output uses vocabulary-sized sparse coordinates. Store it in an engine with sparse-vector support, not a dense-only collection.
Produce ColBERT-style token vectors encode-multi-vector
from sentence_transformers import MultiVectorEncoder
encoder = MultiVectorEncoder("lightonai/GTE-ModernColBERT-v1")
query_vectors = encoder.encode_query(["capital of France"])
document_vectors = encoder.encode_document(["Paris is the capital of France."])
scores = encoder.similarity(query_vectors, document_vectors)Version 6 returns a list of 2D tensors because each input has a different token count. A one-vector-per-row index cannot store this representation directly.
Limit late-interaction scoring memory bound-maxsim-memory
scores = encoder.similarity(
query_vectors,
document_vectors,
chunk_elements=1_000_000,
device="cuda",
)`chunk_elements` bounds the token-pair intermediate. Lower it after an out-of-memory result; scoring returns each completed chunk to the caller device.
Place dense inference on CUDA select-device
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cuda")
embeddings = model.encode(sentences, batch_size=64)This succeeds only with a CUDA-capable PyTorch build and compatible driver. Test an encode call, since package installation alone does not verify the device.
Load a reviewed local model load-offline-model
model = SentenceTransformer("./models/all-MiniLM-L6-v2")
embeddings = model.encode(sentences)Bake or mount the complete model directory and enable Hugging Face offline mode when production must never fall back to a Hub request.
Use a trained Matryoshka dimension truncate-matryoshka
model = SentenceTransformer("mixedbread-ai/mxbai-embed-large-v1", truncate_dim=512)
embeddings = model.encode(sentences)Dimension truncation preserves useful ordering only for models trained for Matryoshka representations. Cutting an arbitrary embedding model discards information.
Choose the ONNX inference extra install-onnx-runtime
python -m pip install 'sentence-transformers[onnx]==6.0.0'The ONNX extra changes the serving dependency set. Export and benchmark the exact model before assuming it improves CPU latency.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| fastembed | PyPI | Use it for smaller ONNX-based embedding and reranking workers that do not need this training stack. |
| transformers | PyPI | Use it directly when the application already owns tokenization, pooling, batching, and similarity behavior. |
| txtai | PyPI | Use it when embeddings should arrive inside a higher-level semantic search and workflow system. |
More ai / ml guides
openai · mcp · huggingface-hub · @modelcontextprotocol/sdk · scikit-learn · tiktoken · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.