docker
The docker package (usually called docker-py) is the official Python SDK for the Docker Engine HTTP API. You call docker.from_env(), get a client, and then drive containers, images, networks, volumes, and Swarm services as Python objects: client.containers.run() to start something, container.logs(stream=True) to follow output, client.images.build() to build from a Dockerfile. Under the hood it is a requests-based HTTP client talking to the Docker daemon over a Unix socket, a named pipe on Windows, TCP, or SSH. There are two layers: the high-level models API with objects and collections, and the low-level docker.APIClient that maps one method to one Engine endpoint when you need parameters the models do not expose.
Still the right library for programmatic container lifecycle management from Python, and the only one that is officially maintained by Docker. Reach for python-on-whales the moment you need BuildKit or compose, and for aiodocker if you are async.
Use it if
- You are automating container lifecycle from Python: CI runners, job schedulers, test suites that spin up a database container, or a control plane that starts and stops workloads on a host
- You want structured results instead of parsing CLI text: container.attrs is the raw inspect JSON, container.wait() returns the exit status dict, and client.containers.list(filters=...) returns objects rather than a table you have to regex
- You need to stream events, logs, or stats continuously; client.events(decode=True) and container.stats(stream=True) give you generators of dicts straight from the daemon
- You are talking to a remote daemon over TCP with TLS or over SSH, where shelling out to a local docker binary is not an option
- You need Engine features the CLI hides, like precise host config flags, custom networking_config on create, or direct Swarm service updates
- You need BuildKit. client.images.build() posts to the classic builder endpoint, so Dockerfile syntax directives, RUN --mount cache mounts, secrets, and multi-platform builds either fail or quietly build the wrong thing. The docker CLI has defaulted to BuildKit for years, so a Dockerfile that builds fine in your terminal can break here. python-on-whales shells out to the real CLI and does not have this gap
- You need Docker Compose. There is no compose support and there never has been; the v2 compose implementation lives in Go inside the CLI plugin. Reimplementing a compose file with individual create calls is a project, not a workaround
- Your application is asyncio. Every call blocks on requests, so a single container.wait() or streaming log loop stalls the event loop unless you push it to a thread pool. aiodocker is the native async client
- All you want is a throwaway Postgres or Redis for tests. testcontainers already wraps this SDK with waiting, port mapping, and cleanup logic you would otherwise write yourself and get subtly wrong
- You depend on tight typing or a fast-moving release cadence. The package ships no inline type hints (community stubs live in types-docker), and 7.1.0 in May 2024 was followed by 7.2.0 only in July 2026, with 445 open issues (564 counting PRs) in the tracker
Setup reality
pip install docker is small, requiring only requests and urllib3 (plus pywin32 on Windows). The friction is everything around the connection. SSH transport needs the extra: pip install docker[ssh] for paramiko. Permissions bite immediately, since docker.from_env() on Linux fails with a permission error unless your user is in the docker group or you point DOCKER_HOST at a rootless socket. As of 7.2.0, from_env() now honors the active Docker CLI context when DOCKER_HOST is unset, which is friendlier but changes where an existing script connects; pass use_context=False or set DOCKER_CONTEXT=default to pin the old behavior. The default request timeout is 60 seconds, so long builds and pulls raise a read timeout unless you raise it at client construction. The default Engine API version is 1.45 with a minimum of 1.24, so an old daemon needs an explicit version= or auto negotiation. Blocking calls leave sockets open, so call client.close() when you are done; there is no context manager on DockerClient.
Patterns
Create a client and verify the connectionconnect-to-daemon
import docker
client = docker.from_env(timeout=120)
client.ping() # True, or raises
print(client.version()["ApiVersion"])
# explicit endpoints instead of the environment:
# docker.DockerClient(base_url="unix:///var/run/docker.sock")
# docker.DockerClient(base_url="ssh://deploy@host") # needs docker[ssh]
client.close()The default timeout is 60 seconds, which is not enough for large pulls or builds. Since 7.2.0 from_env() follows the active Docker CLI context when DOCKER_HOST is unset; pass use_context=False if you want the pre-7.2 behavior.
Run a container to completion and read its outputrun-container-and-get-output
import docker
from docker.errors import ContainerError, ImageNotFound
client = docker.from_env()
try:
out = client.containers.run(
"python:3.12-slim",
["python", "-c", "print(6*7)"],
remove=True,
)
print(out.decode()) # b'42\n'
except ContainerError as e:
print(e.exit_status, e.stderr)
except ImageNotFound:
print("image missing and pull failed")Without detach=True this blocks and returns stdout as bytes, and a nonzero exit raises ContainerError rather than returning the code. remove=True deletes the container afterwards, which also destroys the logs you might want on failure.
Start a long-running container with ports, env, and volumesrun-detached-with-config
container = client.containers.run(
"postgres:16",
detach=True,
name="pg-test",
environment={"POSTGRES_PASSWORD": "secret"},
ports={"5432/tcp": 55432},
volumes={"/tmp/pgdata": {"bind": "/var/lib/postgresql/data", "mode": "rw"}},
labels={"owner": "integration-tests"},
auto_remove=False,
)
print(container.short_id, container.status)ports keys are 'port/protocol' strings on the container side and the value is the host port; getting that backwards silently publishes nothing useful. Host paths in volumes are resolved by the daemon, not your process, so they must exist on the Docker host, which is not your machine when talking to a remote daemon.
Wait for a container and read its exit codewait-and-check-exit
result = container.wait(timeout=300)
print(result["StatusCode"])
container.reload() # refresh .status and .attrs
print(container.status) # 'exited'
print(container.attrs["State"]["OOMKilled"])
logs = container.logs(stdout=True, stderr=True).decode()
container.remove()Container objects are snapshots: .status and .attrs are frozen at the moment you fetched the object, so call reload() before trusting them. Collect logs before remove(), because removal deletes them.
Follow container logs line by linestream-logs
for raw in container.logs(stream=True, follow=True, since=0, timestamps=True):
print(raw.decode(errors="replace").rstrip())
# non-blocking tail instead:
# print(container.logs(tail=100).decode())The generator yields bytes chunks that usually align with lines but are not guaranteed to, so buffer if you parse structured output. With follow=True the loop only ends when the container stops or the connection drops, and the client timeout does not apply to an active stream.
Run a command inside a running containerexec-in-container
res = container.exec_run(
["sh", "-c", "echo out; echo err 1>&2"],
demux=True,
workdir="/app",
environment={"MODE": "check"},
)
stdout, stderr = res.output
print(res.exit_code, stdout, stderr)Pass the command as a list to avoid shell quoting surprises; a plain string is split naively. Without demux=True stdout and stderr come back interleaved in one bytes blob. exec_run does not accept a timeout, so a hung command hangs your thread.
Build an image from a Dockerfilebuild-image
image, build_logs = client.images.build(
path="./service",
dockerfile="Dockerfile",
tag="myorg/service:dev",
buildargs={"VERSION": "1.4.0"},
rm=True,
nocache=False,
)
for chunk in build_logs:
if "stream" in chunk:
print(chunk["stream"], end="")This uses the classic builder, not BuildKit, so a '# syntax=' directive, RUN --mount, --secret, or multi-platform output will not work here even though they work in your terminal. The whole build context directory is tarred and uploaded, so a missing .dockerignore can send gigabytes over the socket.
Pull an image and show progresspull-with-progress
image = client.images.pull("nginx", tag="1.27-alpine")
print(image.tags, image.short_id)
# streaming progress needs the low-level client:
for line in client.api.pull("nginx", tag="1.27-alpine",
stream=True, decode=True):
print(line.get("status"), line.get("progress", ""))The high-level pull blocks with no feedback until it finishes, which looks like a hang on large images. client.api is the low-level APIClient and is always available on a DockerClient when the model layer lacks an option.
Authenticate against a private registryprivate-registry-auth
client.login(username="ci", password=token,
registry="registry.example.com")
client.images.pull("registry.example.com/team/app:latest")
# or per call, skipping stored credentials:
auth = {"username": "ci", "password": token}
client.images.pull("registry.example.com/team/app",
tag="latest", auth_config=auth)client.login() caches the credential in the client only; it does not write to ~/.docker/config.json. If a credential helper is configured on the host, docker-py tries to shell out to it, which fails inside slim containers that do not ship the helper binary.
List and filter containers and imageslist-and-filter
for c in client.containers.list(all=True,
filters={"label": "owner=integration-tests",
"status": "exited"}):
print(c.name, c.image.tags, c.attrs["State"]["ExitCode"])
for img in client.images.list(filters={"dangling": True}):
print(img.id)list() defaults to running containers only; pass all=True or you will miss exited ones. Filters are passed straight to the Engine, so an unsupported key raises an APIError instead of being ignored.
React to daemon events in real timewatch-events
for event in client.events(decode=True,
filters={"type": "container",
"event": ["start", "die"]}):
print(event["Action"], event["Actor"]["Attributes"].get("name"))Without decode=True you get raw JSON bytes per line. The stream runs forever and reconnects are your problem: if the daemon restarts, the generator raises and you lose every event until you reconnect, so record the last timestamp and pass since= on retry.
Prune stopped containers, images, and volumescleanup-resources
print(client.containers.prune()["SpaceReclaimed"])
print(client.images.prune(filters={"dangling": False})["SpaceReclaimed"])
print(client.volumes.prune()["SpaceReclaimed"])
usage = client.df()
print(usage["LayersSize"])images.prune with dangling set to False removes every unused image, not just untagged ones, which on a shared host deletes images other jobs were about to use. Scope destructive pruning with a label filter.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| python-on-whales | PyPI | You need BuildKit, compose, or buildx; it drives the real docker CLI so anything the CLI does works, at the cost of requiring the binary. |
| aiodocker | PyPI | Your service is asyncio and you cannot afford blocking HTTP calls to the daemon inside the event loop. |
| testcontainers | PyPI | You only want disposable service containers for integration tests, with readiness waits and cleanup already handled. |