PyPIAI / MLupdated 05 Aug 2026

transformers

Item: transformers
Rating: 4.5
Author: Keyoor

Hugging Face's model-definition library: the standard way to download, run, and fine-tune pretrained models for text, vision, audio, video, and multimodal tasks in PyTorch. Over one million checkpoints on the Hugging Face Hub load through its Auto classes and pipeline() API, and its model definitions are the common format that vLLM, SGLang, llama.cpp, Axolotl, and most of the training and inference ecosystem build against. v5 targets Python 3.10+ and PyTorch 2.5+.

transformers · v5.14.1 · 47.2M/wk · ★ 163,340 · pushed 05 Aug 2026 · huggingface/transformers · docs · reviewed by Keyoor

Verdict

The center of gravity for open-source ML; nothing else matches its Hub integration and task breadth. Use it to load, experiment, and fine-tune, then hand heavy production serving to a dedicated engine.

API stability4/5The core surface (pipeline, Auto classes, from_pretrained) has survived every major; v5 removed long-deprecated paths and per-model kwargs still shift between releases

Docs4/5Extensive official docs with task guides, API reference, and free courses; at this breadth quality varies by page and older examples drift out of date

Maintenance5/5One of the most active repos on GitHub: 163k stars, pushes within the hour, frequent releases, backed by Hugging Face; 2.3k open issues and PRs is the cost of that scale

Ecosystem5/5Over 1M compatible checkpoints on the Hub; PEFT, TRL, Accelerate, vLLM, and most training frameworks consume its model definitions directly

Use it if

You want to run or fine-tune open-weights models from the Hugging Face Hub in a few lines of Python
You need one API (pipeline, AutoModel, AutoTokenizer) that spans text, vision, audio, and multimodal tasks
You are fine-tuning: Trainer plus the surrounding stack (PEFT, TRL, Accelerate) is the standard tooling
You need the reference model definition that inference engines and export formats are built from

Skip it if

You are serving LLMs in production: plain transformers generation is far slower than dedicated engines like vLLM; prototype here, serve there
You only need text embeddings: sentence-transformers wraps this with a simpler purpose-built API
You run on CPU or edge hardware: GGUF models via llama.cpp bindings are lighter and faster there
You want a small dependency: the install plus PyTorch runs to gigabytes before you download a single model
Your codebase is pinned to TensorFlow or JAX: v5 targets PyTorch, so you would be stuck maintaining aging 4.x pins

Setup reality

pip install "transformers[torch]" in a fresh venv; the heavy lift is PyTorch itself, whose CUDA wheels are multi-gigabyte and must match your driver stack. First use of any model downloads weights into ~/.cache/huggingface, which fills disks fast, and gated models like Llama need a Hub token plus an accepted license before from_pretrained stops throwing 401s. GPU out-of-memory errors are the standard rite of passage: expect to reach for dtype=torch.bfloat16, device_map="auto", and quantization before larger models fit.

Patterns

Generate text with a pipelinepipeline-text-generation

from transformers import pipeline

generator = pipeline(task="text-generation", model="Qwen/Qwen2.5-1.5B")
print(generator("the secret to baking a really good cake is "))

The model downloads and caches on first use; pin an explicit model name because task-only defaults can change between versions.

Chat with an instruct modelchat-with-model

import torch
from transformers import pipeline

chat = [
    {"role": "system", "content": "You are a concise assistant."},
    {"role": "user", "content": "Fun things to do in New York?"},
]

pipe = pipeline(task="text-generation", model="meta-llama/Meta-Llama-3-8B-Instruct",
                dtype=torch.bfloat16, device_map="auto")
out = pipe(chat, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])

Passing a list of role/content dicts applies the model's chat template automatically; Llama models are gated and need a logged-in Hub token.

Load a model and tokenizer directlyload-model-tokenizer

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen2.5-1.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

inputs = tokenizer("Explain ASGI in one line.", return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=60)
print(tokenizer.decode(out[0], skip_special_tokens=True))

In v5 the argument is dtype=, replacing the older torch_dtype= you will see in pre-v5 tutorials.

Apply a chat template manuallyapply-chat-template

messages = [
    {"role": "user", "content": "Write a haiku about compilers."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=60)

Skipping the chat template on an instruct model is the top cause of garbage output; never hand-format the prompt string yourself.

Fit a large model on limited GPU memoryfit-big-model-gpu

import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    dtype=torch.bfloat16,
    device_map="auto",
)

device_map="auto" (via accelerate) shards layers across GPUs and CPU; anything offloaded to CPU runs much slower, so watch the printed device map.

Load a model in 4-bit to halve memory againquantize-4bit

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    quantization_config=bnb,
    device_map="auto",
)

Requires the separate bitsandbytes package and a CUDA GPU; quantized weights cannot be trained directly, only used with adapters like LoRA.

Transcribe audio with Whisperspeech-to-text

from transformers import pipeline

asr = pipeline(task="automatic-speech-recognition", model="openai/whisper-large-v3")
print(asr("meeting.flac")["text"])

Audio decoding needs ffmpeg installed on the system; for files over 30 seconds pass chunk_length_s to avoid truncation.

Classify an imageimage-classification

from transformers import pipeline

clf = pipeline(task="image-classification",
               model="facebook/dinov2-small-imagenet1k-1-layer")
print(clf("parrots.png"))

Accepts local paths, URLs, or PIL images; install pillow if you are not already depending on it.

Save and reload a model locallysave-load-local

model.save_pretrained("./my-model")
tokenizer.save_pretrained("./my-model")

# later, fully offline:
model = AutoModelForCausalLM.from_pretrained("./my-model")
tokenizer = AutoTokenizer.from_pretrained("./my-model")

Set HF_HUB_OFFLINE=1 in air-gapped environments so nothing tries to phone the Hub for version checks.

Chat or serve a model from the command lineserve-cli

# interactive chat in the terminal
transformers chat Qwen/Qwen2.5-0.5B-Instruct

# OpenAI-compatible local server
transformers serve

Handy for smoke-testing a checkpoint before writing code; it is not a production serving stack.

Alternatives

Package	Registry	Pick it when
vllm	PyPI	Production LLM serving where throughput and batching matter
sentence-transformers	PyPI	You only need embeddings for search or similarity
llama-cpp-python	PyPI	CPU or edge inference on quantized GGUF models