pytest-xdist review
pytest-xdist 3.8.0 runs one pytest session through several worker processes, usually on the CPUs of a single machine. Each worker collects the suite, and the controller assigns matching test indexes using schedulers such as load, loadfile, loadscope, or loadgroup. This helps independent CPU-heavy tests; it exposes shared databases, ports, files, and session fixtures that were never isolated. Version 3.8.0 adds --no-loadscope-reorder and a matching ini option, letting loadscope preserve the input file order when that matters more than maximum scheduling freedom. Our Python 3.12 import of xdist completed in 0.52 seconds.
Our pytest-xdist 3.8.0 install took 0.5 seconds and 8 MB, but each of its workers repeats collection and session fixtures. Install it after resources are isolated per worker; leave a short or shared-state suite serial.
We installed it
| Install | ✓ · 0.5s | 7 packages on disk · 8 MB |
| Import | ✓ | import xdist in 0.52s · pure Python · requires Python >=3.9 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does pytest-xdist install cleanly?
Yes. In a fresh container with an empty cache, pip install pytest-xdist finished in 0.5s, leaving 7 packages and 8 MB on disk. pip-audit reported no known vulnerabilities.
What does pytest-xdist need to run?
Python >=3.9, and nothing compiled: it is pure Python. In our run import xdist succeeded in 0.52s.
pytest-xdist or pytest-split: which should you use?
pytest-split: Choose it to divide tests among CI jobs from recorded durations. Our pytest-xdist 3.8.0 install took 0.5 seconds and 8 MB, but each of its workers repeats collection and session fixtures.
When should you not use pytest-xdist?
Tests write to one database, Redis namespace, file, port, or singleton that cannot be divided by worker
Use it if
- A long pytest suite contains independent work that can use several CPU processes
- Each worker can receive its own database, port range, cache, and temporary directory
- Tests sharing an expensive module or class fixture should travel together under loadscope or loadfile
- One CI machine has spare cores and adding more hosted jobs would cost more
- Tests write to one database, Redis namespace, file, port, or singleton that cannot be divided by worker
- You regularly debug with --pdb or uncaptured -s output; distributed mode does not provide the same interactive session
- The suite finishes quickly enough that worker startup and repeated collection dominate the run
- A rate-limited API or small database is already the bottleneck; extra workers add pressure instead of useful CPU work
- Collection depends on sets, unsorted directory reads, random values, or worker-specific environment because every worker must return identical ordered test IDs
Setup reality
We installed pytest-xdist 3.8.0 in 0.5 seconds on Python 3.12. Seven packages used 8 MB on disk, and pip-audit reported 0 known vulnerabilities. The measured package had 5 direct dependencies and pure Python code. import xdist worked in 0.52 seconds. It requires Python 3.9 or newer, does not ship py.typed, and exposed no license value in the installed metadata we inspected.
Installing the plugin registers -n and --dist; no account, service credential, or required config file is involved. PyPI declares pytest >=7 and execnet >=2.1. pytest -n auto follows detected physical CPU availability. A container can see a host count larger than its CPU quota, so a fixed -n value is easier to reason about in CI. Optional extras bring psutil or setproctitle.
Each worker imports conftest.py and owns a pytest session. A session-scoped fixture therefore runs once in every worker. Build resource names from worker_id, which is gw0, gw1, and so on during distribution. testrun_uid is shared across workers for one invocation and can namespace a common external resource. Shared creation still needs a file or service lock because worker startup order is not defined.
All workers must collect the same test IDs in the same order. Unsorted sets, filesystem enumeration, random parametrization, or environment-dependent skips can stop the run before execution. Begin with 2 workers and inspect failures before selecting loadfile or loadscope for fixture reuse. Version 3.8.0's --no-loadscope-reorder preserves file order under loadscope, though that choice can leave a worker idle when one file is much slower.
Patterns
Run a fixed worker count parallel-workers
pytest -n 4
pytest -n auto
pytest -n 0A fixed count respects a known CI quota. -n 0 returns to serial execution for diagnosis.
Choose how tests are grouped scheduler-mode
pytest -n 4 --dist loadfile
pytest -n 4 --dist loadscope
pytest -n 4 --dist loadgroup
pytest -n 4 --dist workstealloadfile and loadscope keep wider fixture owners together. loadgroup only acts on xdist_group marks.
Keep file order with loadscope loadscope-order
pytest -n 4 --dist loadscope --no-loadscope-reorderVersion 3.8.0 added this switch. Preserved order can balance worse than the default reorder.
Give every worker a database worker-database
import pytest
@pytest.fixture(scope='session')
def database_name(worker_id):
suffix = 'serial' if worker_id == 'master' else worker_id
return f'test_app_{suffix}'worker_id is master without distribution and gw0, gw1, and onward with xdist.
Name one distributed test run run-namespace
@pytest.fixture(scope='session')
def bucket_name(testrun_uid):
return f'integration-{testrun_uid}'testrun_uid is identical in every worker for one invocation. Lock shared provisioning.
Keep resource tests on one worker group-tests
@pytest.mark.xdist_group(name='payments')
def test_charge(): ...
@pytest.mark.xdist_group(name='payments')
def test_refund(): ...
# pytest -n 4 --dist loadgroupThe marker changes placement only under the loadgroup scheduler.
Sort parametrized input deterministic-collection
@pytest.mark.parametrize('region', sorted(load_regions()))
def test_region(region):
...A different test ID order in any worker makes the controller reject collection.
Run setup only in the controller controller-only-hook
import xdist
def pytest_configure(config):
if xdist.is_xdist_worker(config):
return
start_controller_service()pytest_configure can execute in the controller and workers. Guard a side effect that should happen once.
Limit crashed-worker retries worker-restarts
pytest -n 4 --max-worker-restart 0
pytest -n 4 --max-worker-restart 2A killed or crashed process may restart. A low ceiling exposes repeated OOM or native crashes.
Reproduce one case serially serial-debug
pytest -n 0 -s --pdb tests/test_orders.py::test_raceSerial mode restores the ordinary interactive debugger and uncaptured terminal behavior.
Collect coverage from workers parallel-coverage
pytest -n 4 --cov=myapp --cov-report=term-missingUse a pytest-cov version that combines worker data, then compare once against -n 0.
Cap automatic workers auto-worker-hook
def pytest_xdist_auto_num_workers(config):
return 4This hook gives the project one ceiling even when developers type -n auto.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| pytest-split | PyPI | Choose it to divide tests among CI jobs from recorded durations. |
| pytest-parallel | PyPI | Evaluate it when you specifically need its thread and process execution model. |
| pytest-forked | PyPI | Choose it for per-test process isolation without xdist scheduling. |
More testing guides
pytest · chai · vitest · jsdom · playwright · coverage · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.