cython
Cython is a compiler that turns Python-like source into C, then hands that C to your normal compiler to produce an importable extension module. It accepts almost all real Python, so an unmodified .py file usually compiles as-is, but the point is the extra syntax on top: you can declare that a variable is a C int, that a function is a C function with no Python call overhead, that a NumPy array is a contiguous buffer of doubles, and that a loop should run with the GIL released. Those declarations are what produce the order-of-magnitude speedups people associate with it. Cython also works in the other direction, as the standard way to call an existing C or C++ library from Python: you write the header declarations once and get a normal Python module out. Two dialects exist. The classic one is .pyx files with cdef and cpdef, and the newer pure Python mode keeps everything in a valid .py file using type annotations and a cython shadow module, so the same file still runs uncompiled under plain CPython.
Cython is still the right tool when you are wrapping C or squeezing a profiled hot loop, and pure Python mode has removed most of the excuse for not trying it. Just be clear that you are adopting a compiler and a wheel-building pipeline, not adding a dependency, and check Numba first if all you have is NumPy math.
Use it if
- You are wrapping an existing C or C++ library and want a Python module with real classes and exceptions rather than a ctypes translation layer you maintain by hand
- You have profiled and found a specific hot loop over numeric data: typed memoryviews plus boundscheck=False turn a per-element Python object dance into a plain C array walk
- You need actual parallelism inside one process: prange with nogil=True runs an OpenMP loop with the GIL released, which no amount of threading in pure Python will give you on a non-free-threaded interpreter
- You maintain a library where compiling at install time is already normal and you ship wheels anyway. NumPy, SciPy, pandas, scikit-learn, lxml, and uvloop all take this route
- You want the compiled and interpreted versions to stay the same code: pure Python mode keeps the file importable, testable, and debuggable as ordinary Python while the release build compiles it
- You have not profiled, and you are hoping compilation alone makes your app faster. Compiling untyped Python removes interpreter dispatch overhead and little else; the big numbers in the tutorials come from adding C types to specific loops, which is a rewrite of that code, not a build flag
- Your hot code is array math. Numba's @njit gives you comparable numeric speed with no build step, no compiler on the user's machine, and no wheel matrix, and you can delete the decorator to get plain Python back. Cython wins on wrapping C and on non-numeric code, but for a NumPy kernel Numba is far less operational work
- You are not ready to own a build pipeline. From the day you add a .pyx file, every user needs a C toolchain unless you publish wheels for every OS, architecture, and Python version, now including free-threaded builds. cibuildwheel becomes a permanent part of your CI, and any sdist install on a machine without a compiler fails with a wall of C errors
- Debuggability matters more than speed. Tracebacks point into generated C, pdb will not step into a cdef function without setting up cygdb, coverage needs a dedicated plugin and a compile-time flag, and the generated .c file for a modest module is often tens of thousands of lines
- You want mistakes to raise exceptions. Once you turn off boundscheck and wraparound (which is where most of the speed lives), an off-by-one reads past the end of the buffer and you get corrupted numbers or a segfault instead of an IndexError. cdivision=True also swaps Python's floor division and ZeroDivisionError for C truncation and undefined behavior
- You are allergic to a large open bug queue. There are 1331 open issues (1501 counting PRs). That is proportional to a compiler with two decades of language surface rather than a sign of neglect, but it does mean your specific edge case may already be filed and unresolved
Setup reality
pip install Cython pulls no Python dependencies but it is useless without a C compiler on the machine that builds: Xcode Command Line Tools on macOS, the MSVC Build Tools on Windows, and gcc or clang plus the python3-dev headers on Linux, which is the single most common reason a first build fails. Cython itself must go in build-system.requires in pyproject.toml, never in your runtime dependencies, and your setup.py has to call cythonize() rather than listing the .pyx directly. Always pin language_level ("3str" or 3) in cythonize or a directive comment, because leaving it unset prints a warning on every build and leaves the default open to change. Generated .c files should be gitignored but you must decide deliberately whether the sdist contains them: shipping them means users do not need Cython installed, but a .c file generated against an older release will not compile against a newly released CPython, so a source install breaks the week Python 3.15 lands. Editable installs do not rebuild when you edit a .pyx, so you will chase a stale .so at least once. OpenMP needs explicit extra_compile_args and extra_link_args, and Apple's clang has no OpenMP support out of the box, so prange silently compiles to a serial loop or fails to link until you install libomp. If you cimport numpy you also need numpy's headers via include_dirs and numpy as a build requirement, which is separate from having numpy installed at runtime.
Patterns
Build your first extension modulecompile-a-pyx-module
# pkg/fast.pyx
def fib(int n):
cdef int a = 0, b = 1, i
for i in range(n):
a, b = b, a + b
return a
# setup.py
from setuptools import setup, Extension
from Cython.Build import cythonize
extensions = [Extension("pkg.fast", ["pkg/fast.pyx"], extra_compile_args=["-O3"])]
setup(ext_modules=cythonize(extensions, language_level="3str", annotate=True))
# build
pip install -e .
python -c "from pkg.fast import fib; print(fib(50))"The Extension name must be the full dotted import path or the built .so lands somewhere Python will not find it. Pass the .pyx to cythonize, never to Extension alone, or setuptools will try to hand the .pyx straight to the C compiler. Editable installs do not rebuild on edit, so re-run pip install -e . after every change to the .pyx.
Put Cython in build requires, not install requiresdeclare-build-requirements
# pyproject.toml
[build-system]
requires = ["setuptools>=64", "Cython>=3.2"]
build-backend = "setuptools.build_meta"
[project]
name = "pkg"
version = "0.1.0"
dependencies = [] # Cython does NOT belong hereCython is a build-time tool; the compiled module has no runtime dependency on it. Listing it in project.dependencies installs a compiler toolchain into every production environment for nothing. Pin a lower bound that matches the syntax you use, since a user's build environment may resolve to something older than your development machine.
Type the hot loop with cdef and cpdefadd-c-types
cdef double _dot(double[::1] a, double[::1] b) nogil:
cdef Py_ssize_t i
cdef double total = 0.0
for i in range(a.shape[0]):
total += a[i] * b[i]
return total
cpdef double dot(double[::1] a, double[::1] b):
if a.shape[0] != b.shape[0]:
raise ValueError("length mismatch")
with nogil:
return _dot(a, b)cdef functions are C-only and invisible from Python, which is what makes them cheap to call; cpdef generates both a C version and a Python wrapper, so you pay a small size cost for importability. Use Py_ssize_t for indices rather than int: on a 64-bit platform an int loop counter silently overflows past roughly two billion elements.
Keep the file valid Python and still compile itpure-python-mode
# clamp.py -- runs uncompiled under CPython, compiles under Cython
import cython
@cython.cfunc
@cython.exceptval(-1, check=False)
def _clamp(x: cython.int, lo: cython.int, hi: cython.int) -> cython.int:
if x < lo:
return lo
return hi if x > hi else x
def clamp_all(values: list) -> list:
i: cython.Py_ssize_t
return [_clamp(v, 0, 255) for v in values]
if cython.compiled:
print("running the compiled build")The cython module ships a pure-Python shadow implementation, so the decorators and cython.int annotations are no-ops when the file is imported without compiling. That means one file you can unit-test and step through in pdb during development and compile for release. The catch: your linters and type checkers see cython.int as a real annotation and will complain, and cython.compiled is the only reliable way to prove which build you are actually running.
Walk a NumPy array as a C buffertyped-memoryviews
# rows.pyx
# cython: boundscheck=False, wraparound=False, language_level=3str
import numpy as np
def rowsum(double[:, ::1] a):
cdef Py_ssize_t i, j, n = a.shape[0], m = a.shape[1]
out = np.empty(n, dtype=np.float64)
cdef double[::1] o = out
cdef double s
for i in range(n):
s = 0.0
for j in range(m):
s += a[i, j]
o[i] = s
return outThe ::1 says the last axis is C-contiguous, which is what lets Cython emit direct pointer arithmetic; pass a transposed or sliced array and you get a ValueError about a non-contiguous buffer at call time. dtype must match exactly, so a float32 array into a double[] view raises rather than converting. You only need cimport numpy (and numpy's headers as a build dependency) if you touch the NumPy C API; plain memoryviews do not require it.
Turn off the safety checks deliberatelyset-compiler-directives
# per file, first lines of the .pyx
# cython: language_level=3str, boundscheck=False, wraparound=False
# cython: cdivision=True, initializedcheck=False
# per function, when the rest of the module should stay safe
import cython
@cython.boundscheck(False)
@cython.wraparound(False)
def inner(double[::1] a):
...
# or globally in setup.py
# cythonize(exts, compiler_directives={"language_level": "3str", "boundscheck": False})This is where most of the speed comes from and also where the crashes come from. boundscheck=False means an out-of-range index reads foreign memory instead of raising IndexError. wraparound=False makes a[-1] read before the start of the buffer. cdivision=True gives C semantics, so -7 // 2 becomes -3 rather than -4 and dividing by zero is undefined instead of a ZeroDivisionError. Scope them to the function you measured, not the whole file.
Generate the HTML report that shows Python overheadannotate-to-find-slow-lines
cython -a --3str pkg/fast.pyx
open pkg/fast.html # xdg-open on Linux
# or always, from setup.py
# cythonize(extensions, annotate=True)Every line gets a yellow tint proportional to how much CPython C-API work it generates; white lines are pure C. Click a line to expand the generated C underneath it. This is the single most useful tool in the project and the fastest way to discover that your typed loop is still boxing an integer every iteration because one variable stayed untyped.
Run a parallel loop with prangerelease-the-gil
# par.pyx
from cython.parallel import prange
def total(double[::1] a):
cdef Py_ssize_t i
cdef double s = 0.0
for i in prange(a.shape[0], nogil=True, schedule="static"):
s += a[i]
return s
# setup.py
# Extension("par", ["par.pyx"],
# extra_compile_args=["-fopenmp"], extra_link_args=["-fopenmp"])Nothing inside a nogil block may touch a Python object, so a stray print or a list append is a compile error rather than a runtime one, which is the good news. The bad news is platform support: Apple's bundled clang has no OpenMP, so on macOS you need libomp and the right flags or prange quietly degrades to a serial loop. Cython recognises the s += pattern as a reduction; an arbitrary shared write is a data race it will not warn you about.
Declare C headers and own the pointer lifetimewrap-a-c-library
# wrap.pyx
cdef extern from "math.h":
double c_sqrt "sqrt" (double x) nogil
cdef extern from "mylib.h":
ctypedef struct ctx_t:
int flags
ctx_t* ctx_new(int flags)
void ctx_free(ctx_t* c)
int ctx_run(ctx_t* c, const char* cmd)
cdef class Ctx:
cdef ctx_t* _p
def __cinit__(self, int flags=0):
self._p = ctx_new(flags)
if self._p is NULL:
raise MemoryError("ctx_new failed")
def run(self, str cmd):
cdef bytes b = cmd.encode("utf-8")
if ctx_run(self._p, b) != 0:
raise RuntimeError(cmd)
def __dealloc__(self):
if self._p is not NULL:
ctx_free(self._p)Allocate in __cinit__ and free in __dealloc__: __init__ can be skipped by subclasses or called twice, __cinit__ cannot. Keep the bytes object in a named local before passing the char*, because encoding inline creates a temporary that can be freed while C still holds the pointer. You must also add the library to the Extension with libraries= and include_dirs=; cdef extern only tells Cython the signatures, it does not link anything.
cdef class for objects allocated in Cdefine-an-extension-type
cdef class Point:
cdef public double x, y # readable and writable from Python
cdef readonly double norm # Python can read, only C can write
cdef double _scratch # invisible to Python entirely
def __init__(self, double x, double y):
self.x = x
self.y = y
self.norm = (x * x + y * y) ** 0.5
cpdef Point scaled(self, double k):
return Point(self.x * k, self.y * k)Attributes must be declared up front, so instances have no __dict__ and you cannot set an unexpected attribute or monkeypatch a method. That is exactly why they are small and fast, and exactly why unittest.mock.patch.object on a cdef method raises TypeError. Add cdef dict __dict__ if you really need dynamic attributes, which gives back most of the memory you saved.
Iterate with the Jupyter magic or cython.compilecompile-in-a-notebook
%load_ext Cython
%%cython -a --compile-args=-O3
def fib(int n):
cdef int a = 0, b = 1, i
for i in range(n):
a, b = b, a + b
return a
# or, in a plain script, compile one function at import time
import cython
@cython.compile
def square(x: cython.int) -> cython.int:
return x * x%%cython -a renders the annotation report inline, which makes the notebook the fastest place to try typing strategies. @cython.compile invokes a real C compiler the first time the module is imported, so it needs a toolchain on the machine that runs the code, not just the machine that built it. Neither belongs in a production import path.
Build binaries so users never see a compilership-wheels
pip install cibuildwheel
CIBW_BUILD="cp310-* cp311-* cp312-* cp313-* cp313t-*" \
CIBW_SKIP="*-musllinux_i686" \
cibuildwheel --output-dir wheelhouse
# include the generated C in the sdist so a source install needs no Cython
# MANIFEST.in
# include pkg/*.pyx pkg/*.pxd pkg/*.cPublishing an sdist without wheels means every user needs a compiler and matching headers, and that is where most of your bug reports will come from. Shipping the generated .c inside the sdist avoids requiring Cython at install time, but C generated by an older release will not compile against a newly released CPython, so you still have to cut a release each time a new interpreter version appears. Free-threaded builds (the t suffix) need their own wheels and their own testing.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| numba | PyPI | The hot code is NumPy array math and you want a decorator with no build step, no compiler at install time, and no wheel matrix. |
| pybind11 | PyPI | You are binding a C++ library and would rather write modern C++ with templates than learn a second Python-shaped dialect. |
| nuitka | PyPI | You want a whole application compiled and optionally packaged into a standalone binary, rather than one hot module accelerated. |
| pythran | PyPI | You have self-contained scientific Python functions and want them compiled from ordinary annotations, including as a Cython backend for NumPy code. |