sentence-transformers
sentence-transformers (also known as SBERT) is the standard Python framework for running and training text embedding models locally. It loads any of 15,000+ pretrained models from the Hugging Face Hub by name, turns sentences into dense vectors with model.encode(), and also covers Cross-Encoder rerankers and sparse encoders like SPLADE. It sits on top of PyTorch and transformers and is maintained by Hugging Face, which took over the project from UKP Lab.
The default way to run and train embedding models locally, and Hugging Face maintaining it has kept it current (sparse encoders, ONNX paths, multimodal). Respect the heavy dependency chain; for lightweight inference-only jobs, smaller tools do it with a fraction of the install.
Use it if
- You are building semantic search, RAG retrieval, clustering, or deduplication and want local embeddings in a few lines
- You need a reranker: CrossEncoder models score query-passage pairs for a second retrieval stage
- You want to fine-tune an embedding model on your own data with the built-in trainer and its 20+ loss functions
- You want to pick models straight off the MTEB leaderboard and swap them by changing one string
- You cannot afford the install: it requires torch, transformers, scikit-learn, and scipy, which lands in the multi-gigabyte range once CUDA wheels are involved
- You only need inference-time embeddings on CPU with a small footprint; fastembed (ONNX) or model2vec (static vectors) are far lighter
- You would rather pay per token than run models; a hosted embedding API skips the dependency chain, model downloads, and GPU questions entirely
- You need low-latency CPU serving of full transformer encoders; without the onnx or openvino extras, encode() on CPU will disappoint you
Setup reality
pip install sentence-transformers looks like one line but resolves torch, transformers, scikit-learn, and scipy; on GPU machines you often need to install the CUDA-matched torch build first or you silently get the CPU wheel. First use of any model downloads its weights from the Hugging Face Hub into a local cache, so cold starts, air-gapped machines, and Docker images all need planning. Real features hide behind extras: [train] for fine-tuning, [onnx] or [openvino] for fast CPU inference, [image] for multimodal.
Patterns
Encode sentences into embeddingsencode-sentences
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium.",
]
embeddings = model.encode(sentences)
print(embeddings.shape) # (3, 384)The first call downloads the model from the Hugging Face Hub into ~/.cache; budget for that in Docker builds and cold starts.
Compute similarity between embeddingscompute-similarity
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6660, 0.1046], ...])model.similarity() uses the similarity function the model was trained with (usually cosine), so prefer it over rolling your own dot products.
Search a corpus with query and document embeddingssemantic-search
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
corpus_emb = model.encode(corpus, convert_to_tensor=True)
query_emb = model.encode("How big is Berlin?", convert_to_tensor=True)
hits = util.semantic_search(query_emb, corpus_emb, top_k=5)[0]
for hit in hits:
print(hit["score"], corpus[hit["corpus_id"]])util.semantic_search is fine to a few hundred thousand docs; beyond that move the vectors into a real vector index.
Rerank passages with a Cross-Encoderrerank-cross-encoder
from sentence_transformers import CrossEncoder
model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
ranks = model.rank("How many people live in Berlin?", passages, return_documents=True)
for r in ranks:
print(f"{r['score']:.2f}", r["text"])Cross-Encoders score every query-passage pair, so they are accurate but slow; use them on a shortlist, not the whole corpus.
Run the model on a GPUuse-gpu
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cuda")
embeddings = model.encode(sentences, batch_size=64)If torch was installed as the CPU wheel, device='cuda' fails; install the CUDA-matched torch build before this package.
Normalize embeddings for dot-product searchnormalize-embeddings
embeddings = model.encode(
sentences,
normalize_embeddings=True,
)With unit-length vectors, dot product equals cosine similarity, which is what most vector databases expect as their fast path.
Encode a large corpus in batches with a progress barbatch-encode-progress
embeddings = model.encode(
corpus,
batch_size=128,
show_progress_bar=True,
convert_to_numpy=True,
)Bigger batch_size helps on GPU until you hit out-of-memory; on CPU it changes little, the model is the bottleneck.
Save a model locally and load it offlinesave-load-local
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
model.save("./models/all-MiniLM-L6-v2")
# later, no network needed:
model = SentenceTransformer("./models/all-MiniLM-L6-v2")Loading from a local path avoids Hub calls at startup; set HF_HUB_OFFLINE=1 to guarantee nothing reaches the network.
Shrink embeddings with Matryoshka truncationtruncate-dimensions
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"mixedbread-ai/mxbai-embed-large-v1",
truncate_dim=512,
)
embeddings = model.encode(sentences)
print(embeddings.shape[1]) # 512Only models trained with Matryoshka loss keep quality when truncated; cutting an ordinary model's dimensions just loses information.
Generate sparse embeddings with a SparseEncodersparse-embeddings
from sentence_transformers import SparseEncoder
model = SparseEncoder("naver/splade-cocondenser-ensembledistil")
embeddings = model.encode(sentences)
print(embeddings.shape) # (3, 30522), vocabulary-sized
stats = SparseEncoder.sparsity(embeddings)
print(f"Sparsity: {stats['sparsity_ratio']:.2%}")SparseEncoder arrived in v5; sparse vectors are vocabulary-sized, so store them in an engine with sparse support rather than a dense index.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| fastembed | PyPI | When you want ONNX-based embedding inference without installing torch |
| model2vec | PyPI | When you need very fast static embeddings on CPU and can trade some accuracy |
| transformers | PyPI | When you already depend on it and are willing to write your own pooling and similarity code |