ddtrace
ddtrace is Datadog's Python client for application performance monitoring. You install it, run your app under the ddtrace-run wrapper (or import ddtrace.auto as the first line of your entrypoint), and it monkeypatches roughly a hundred common libraries at import time so that every Django view, Flask route, psycopg query, requests call, Celery task, and boto3 API call becomes a span in a distributed trace. The same package also carries the continuous profiler, exception replay, dynamic instrumentation, test optimization for pytest, the AppSec and IAST security products, and LLM Observability for tracing OpenAI or Anthropic calls. Spans go to a Datadog Agent over localhost port 8126, and the Agent forwards them to Datadog. Nothing here is a general-purpose tracing library: it is the client half of a paid product.
If you pay Datadog, this is the only sane way to instrument Python and it does a lot for one wrapper command. If you do not pay Datadog, or if you want the freedom to move backends later, start with OpenTelemetry instead.
Use it if
- Your company already pays for Datadog and you want traces, profiles, and log correlation from a Python service without writing instrumentation by hand
- You run a Django, Flask, FastAPI, Celery, or aiohttp app built out of well-known libraries and want database, cache, and HTTP client spans to appear with zero code changes
- You need distributed trace context to flow across services and queues, including into and out of non-Python services that also run Datadog tracers
- You want Datadog LLM Observability for an AI feature and would rather use the shipped decorators and provider integrations than build your own span schema
- You want continuous CPU and memory profiling in production, turned on with an environment variable rather than a second agent
- You do not have a Datadog contract: this library only emits to a Datadog Agent, so without the paid backend it collects data that goes nowhere
- You want vendor neutrality: opentelemetry-sdk with the OTLP exporter gives you the same auto-instrumentation idea while letting you swap backends, and ddtrace ties your instrumentation to one vendor's span schema
- You cannot tolerate import-time monkeypatching: ddtrace patches libraries out from under you, which interacts badly with gevent monkeypatching order, uWSGI and Gunicorn preforking, and anything that reimports modules, and the failure mode is a hung worker or missing traces rather than a clean error
- You are size or startup sensitive: the Linux x86_64 wheel for 4.12.2 is about 8.4 MB of compiled extensions, and instrumenting a hundred modules adds real time to cold starts on Lambda and short-lived jobs
- You are on Python 3.8 or older, or already moved to 3.15: 4.x requires Python 3.9 or newer and the current metadata caps support below 3.15
- You upgrade dependencies rarely: Datadog ships a new minor roughly every two weeks and moves public API around between majors, so a two-year-old pin means an awkward migration when you finally need a fix
Setup reality
pip install ddtrace fetches a prebuilt wheel on common Linux and macOS targets, so the compiled Cython and Rust pieces are usually not your problem. Everything after that is. You need a Datadog Agent reachable at localhost:8126 or a DD_TRACE_AGENT_URL pointing at one, and until that exists the library silently buffers and drops. Instrumentation only happens if ddtrace loads before the libraries it patches, which means the ddtrace-run wrapper or import ddtrace.auto on line one of your entrypoint, not somewhere in your settings module. Using both ddtrace-run and ddtrace.auto together is explicitly unsupported. Service naming is environment-driven: DD_SERVICE, DD_ENV, and DD_VERSION are what make the Datadog UI usable, and skipping them leaves everything filed under a guessed name. Under Gunicorn or uWSGI with preforking, the writer thread has to survive the fork, and gevent users have to let gevent patch first. Expect to read Datadog's product docs, not just the ReadTheDocs API reference, because most real configuration lives in environment variables documented on the Datadog site.
Patterns
Instrument an app without touching its coderun-with-ddtrace-run
# shell
export DD_SERVICE=checkout-api
export DD_ENV=prod
export DD_VERSION=2026.08.01
export DD_TRACE_AGENT_URL=http://datadog-agent:8126
ddtrace-run gunicorn -w 4 -k uvicorn.workers.UvicornWorker app:apiddtrace-run sets a sitecustomize hook so instrumentation loads before your imports. Set DD_SERVICE, DD_ENV, and DD_VERSION or every service shows up under a guessed name and you lose deployment tracking.
Instrument from inside the entrypoint insteadimport-ddtrace-auto
# app.py
import ddtrace.auto # must be the very first import
from fastapi import FastAPI
import httpx
api = FastAPI()
@api.get("/health")
async def health():
async with httpx.AsyncClient() as client:
await client.get("https://example.com/ping")
return {"ok": True}Use this when you cannot control the launch command (containers with fixed entrypoints, Lambda handlers). Combining it with ddtrace-run is unsupported and produces double patching.
Add a span around code no integration coversmanual-span
from ddtrace.trace import tracer
def price_basket(basket):
with tracer.trace("pricing.calculate", service="checkout-api", resource="price_basket") as span:
span.set_tag("basket.items", len(basket.items))
span.set_tag("basket.currency", basket.currency)
total = sum(line.amount for line in basket.items)
span.set_metric("basket.total", total)
return totalImport the tracer from ddtrace.trace, not from ddtrace directly; the top-level re-export is on its way out. set_tag takes strings for faceting, set_metric takes numbers you want to aggregate.
Trace a function with a decoratordecorate-function
from ddtrace.trace import tracer
@tracer.wrap("report.render", service="reporting", resource="monthly_pdf")
def render_monthly_pdf(account_id: str) -> bytes:
...
@tracer.wrap()
async def refresh_cache():
...tracer.wrap handles sync, async, generator, and async generator functions. With no arguments the span name is derived from the module and function name, which is usually fine for internal helpers and useless on a dashboard.
Mark a span as failed with the exception attachedrecord-error-on-span
from ddtrace.trace import tracer
with tracer.trace("payment.charge") as span:
try:
gateway.charge(order)
except GatewayTimeout:
span.set_traceback()
span.set_tag("payment.outcome", "timeout")
raise
except CardDeclined as exc:
# expected business outcome, do not flag the span red
span.set_tag("payment.outcome", "declined")
span.set_tag("payment.decline_code", exc.code)An exception that escapes the with block is recorded automatically. The useful move is the opposite one: swallowing expected failures like a declined card so your error rate reflects real problems.
Patch only the libraries you wantselective-patching
import ddtrace
ddtrace.patch(
django=True,
psycopg=True,
redis=True,
requests=False,
logging=True,
)
# equivalently, without touching code:
# DD_TRACE_REQUESTS_ENABLED=false ddtrace-run ...Call patch before importing the target libraries or it does nothing. Turning integrations off one at a time is how you isolate the one that breaks under gevent or a preforking server.
Get trace IDs into your log linescorrelate-logs
# shell
export DD_LOGS_INJECTION=true
# or, for a custom formatter:
import logging
from ddtrace.trace import tracer
class TraceContextFilter(logging.Filter):
def filter(self, record):
ctx = tracer.get_log_correlation_context()
record.dd_trace_id = ctx.get("trace_id", "0")
record.dd_span_id = ctx.get("span_id", "0")
return TrueDD_LOGS_INJECTION only rewrites records going through the stdlib logging module. If you use structlog or write JSON yourself, add the fields via get_log_correlation_context or the trace-to-log jump silently does nothing.
Keep all the errors and a slice of everything elsesampling-rules
# shell
export DD_TRACE_SAMPLING_RULES='[
{"service": "checkout-api", "name": "django.request", "sample_rate": 1.0},
{"service": "checkout-api", "resource": "GET /healthz", "sample_rate": 0.0},
{"sample_rate": 0.1}
]'Rules are evaluated in order and the first match wins, so the catch-all belongs last. Dropping health checks to 0.0 is usually the single biggest ingestion-bill saving available.
Turn on the continuous profilerenable-profiler
# shell
export DD_PROFILING_ENABLED=true
export DD_PROFILING_TIMELINE_ENABLED=true
ddtrace-run python -m myapp
# or in code, when ddtrace-run is not available:
import ddtrace.profiling.auto # noqa: F401The profiler runs its own sampling threads inside your process, so measure the overhead on a canary before enabling it fleet-wide. It needs the same Datadog Agent as tracing.
Trace an LLM call chainllm-observability
from ddtrace.llmobs import LLMObs
from ddtrace.llmobs.decorators import workflow, task
LLMObs.enable(ml_app="support-copilot", agentless_enabled=True)
@task
def fetch_context(question: str) -> list[str]:
return vector_store.search(question, k=5)
@workflow
def answer(question: str) -> str:
chunks = fetch_context(question)
reply = openai_client.chat.completions.create(...) # traced automatically
LLMObs.annotate(tags={"channel": "web"})
return reply.choices[0].message.contentProvider SDK calls are captured by the normal integrations once LLMObs is enabled; the decorators exist to group them into a workflow you can read. agentless_enabled=True skips the local Agent and needs DD_API_KEY set.
Keep the tracer out of local runs and CIdisable-in-tests
# .env.test / CI config
DD_TRACE_ENABLED=false
DD_PROFILING_ENABLED=false
DD_INSTRUMENTATION_TELEMETRY_ENABLED=false
# pytest, when something still imports ddtrace
import pytest
from ddtrace.trace import tracer
@pytest.fixture(autouse=True)
def _no_traces():
tracer.enabled = False
yieldDD_TRACE_ENABLED=false still imports and patches, it just stops sending. If you want the patching gone too, do not run under ddtrace-run in that environment.
Carry trace context across a queue boundarypropagate-context-manually
from ddtrace.propagation.http import HTTPPropagator
from ddtrace.trace import tracer
# producer
headers: dict[str, str] = {}
HTTPPropagator.inject(tracer.current_span().context, headers)
queue.publish(body=payload, headers=headers)
# consumer
context = HTTPPropagator.extract(message.headers)
tracer.context_provider.activate(context)
with tracer.trace("jobs.process", resource=message.job_type):
handle(message)Only needed for transports ddtrace does not already instrument. current_span() returns None when nothing is active, so guard it in code paths that run outside a request.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| opentelemetry-sdk | PyPI | You want vendor-neutral tracing you can point at Datadog, Jaeger, Grafana, or anything else that speaks OTLP |
| sentry-sdk | PyPI | Errors are your main need and tracing is secondary; Sentry costs less and its Python integration is lighter |
| elastic-apm | PyPI | Your observability stack is Elasticsearch and Kibana rather than Datadog |
| newrelic | PyPI | Your company standardized on New Relic; the agent model and tradeoffs are nearly identical |