PyPIInfraupdated 08 Aug 2026

watchtower

Item: watchtower
Rating: 4
Author: Keyoor

Watchtower is a Python logging.Handler that sends standard logging records directly to Amazon CloudWatch Logs through boto3. It queues records and batches PutLogEvents calls instead of making a network request for every line, with a default delivery deadline of 60 seconds. It can create groups and streams, format dictionaries as JSON, tag groups, and derive stream names from logger, process, machine, and time fields. This is application-to-CloudWatch transport, not the Docker container updater with the same name.

watchtower · v3.4.0 · 6.1M/wk · ★ 821 · pushed 24 Nov 2025 · kislyuk/watchtower · docs · reviewed by Keyoor

Verdict

Useful for long-running Python services that truly lack a CloudWatch log collector. If the runtime already captures stdout, use structured stdout instead and remove AWS networking, credentials, buffering, and stream contention from the application.

API stability4/5Watchtower follows semantic versioning and its central contract remains the standard logging.Handler interface plus keyword configuration on CloudWatchLogHandler. Group, stream, queue, interval, formatter, and boto3-client settings have been stable across the current major. Cloud-provider behavior and IAM changes still affect deployments, and the project itself recommends pinning the package for applications or at least the major for libraries.

Docs5/5The README is a full operations guide, not just a synopsis. It documents boto3 credential discovery, recommended IAM policy, the extra permission needed for tags, Flask and Django wiring, YAML dictConfig, custom profiles, stream-name contention across processes, recursion filtering for boto3 dependencies, querying examples, Lambda incompatibility, version pinning, and the default 60-second batching behavior. Those are exactly the failure modes a logging transport needs to disclose.

Maintenance3/5Version 3.4.0 was uploaded in February 2025, the repository's last push was November 2025, and GitHub showed 23 combined open issues and pull requests for a repository with about 820 stars. It is not archived, supports Python 3.8 through 3.13, carries Apache-2.0, and depends on a current boto3 range. Activity is adequate for a mature adapter, but releases are not frequent enough to assume fast responses to AWS behavior changes.

Ecosystem4/5Its use of Python's logging.Handler API makes it compatible with Flask, Django, dictConfig, filters, formatters, and existing library loggers, while boto3 supplies the standard AWS credential chain. CloudWatch itself provides Insights, metric filters, alarms, and exports after ingestion. The ecosystem score is limited by the narrow direct-to-AWS design: modern container platforms generally favor stdout plus an agent or platform logging driver.

Use it if

A long-running Python service must write directly to CloudWatch Logs and no host or container log collector is available
You want to keep using the standard logging module and attach CloudWatch as one handler among console or file handlers
You can provide a least-privilege AWS role and tolerate asynchronous batch delivery
You need deterministic group and per-process stream naming from logger metadata

Skip it if

You run on AWS Lambda: the README says Watchtower is unnecessary there and its background delivery cannot function correctly when Lambda freezes the execution environment
Your platform already ships stdout or stderr to CloudWatch, as ECS, EKS, Docker logging drivers, system agents, and many PaaS setups do: a direct SDK handler duplicates transport and adds credentials to the app
You cannot accept possible loss during hard crashes: records are queued and batched with a 60-second default deadline, so a killed process may end before its pending batch is delivered
Many worker processes will share one log stream: the documentation warns that sequence-token contention grows until events can fail and be dropped, and recommends unique streams per source
You need a cloud-neutral logging path: this handler depends on boto3 and CloudWatch permissions, group behavior, quotas, API availability, and costs

Setup reality

pip install watchtower also brings boto3, but successful import is the easy part. The running process needs AWS credentials discoverable by boto3 and CloudWatch Logs permissions for creating groups or streams, describing streams, and putting events. Prefer an EC2, ECS, EKS, or other workload role over static access keys. Adding log_group_tags also requires logs:TagResource; the older logs:TagLogGroup action is not what current Watchtower uses. Decide whether production is allowed to create log groups and set create_log_group accordingly, then provision retention and encryption outside this handler because creating a group does not establish your organization's lifecycle policy. Delivery is buffered on a queue with a 60-second default send deadline. Call flush or close during an orderly shutdown and understand that SIGKILL, a crash, or a frozen runtime can still strand records. Keep the default stream template or include process_id when using process pools. The README warns that multiple processes sharing a stream cause sequence-token synchronization work and can eventually drop logs with a stderr warning. Django's development server has a separate trap: several framework loggers can deadlock with the handler's threading, so the example disables propagation for them in development. Avoid recursion by not routing boto3, botocore, or urllib3 debug output back through the same handler; Watchtower installs a filter on itself, but other handlers still see those records. Test credentials, region, IAM, shutdown flushing, group retention, network failure, and log volume before relying on it for incident evidence. On Lambda, do not install it for logging at all; write to stdout or stderr and let the platform ship the lines.

Patterns

Attach a CloudWatch handleradd-cloudwatch-handler

import logging
import watchtower

logger = logging.getLogger('orders')
logger.setLevel(logging.INFO)
logger.addHandler(watchtower.CloudWatchLogHandler(log_group_name='orders'))
logger.info('worker started')

With no explicit boto3 client, the standard AWS credential and region chain must resolve in the running environment.

Control AWS region with a boto3 clientuse-explicit-client

import boto3
import watchtower

logs = boto3.client('logs', region_name='eu-west-1')
handler = watchtower.CloudWatchLogHandler(
    boto3_client=logs,
    log_group_name='orders',
)

Prefer a workload role; passing a client should not become an excuse to hard-code access keys in application code.

Give each worker process its own streamname-worker-streams

handler = watchtower.CloudWatchLogHandler(
    log_group_name='orders',
    log_stream_name='{machine_name}/{program_name}/{logger_name}/{process_id}',
)

The documentation warns that independent processes sharing one stream contend on sequence tokens and can eventually drop events.

Set the maximum batching delaytune-delivery-window

handler = watchtower.CloudWatchLogHandler(
    log_group_name='orders',
    send_interval=10,
)

The default delivery deadline is 60 seconds; a shorter interval increases API traffic and still cannot protect against SIGKILL.

Send a dictionary as structured JSONlog-structured-event

logger.addHandler(handler)
logger.info({
    'event': 'order.accepted',
    'order_id': 'ord_123',
    'amount_cents': 2599,
})

Keep secrets and personal data out before the record reaches the handler; CloudWatch retention and access are separate controls.

Tag a managed log grouptag-log-group

handler = watchtower.CloudWatchLogHandler(
    log_group_name='orders',
    log_group_tags={'service': 'orders', 'environment': 'production'},
)

Tagging requires logs:TagResource in addition to write permissions; logs:TagLogGroup is the older API and is not used.

Require infrastructure to create the groupuse-precreated-group

handler = watchtower.CloudWatchLogHandler(
    log_group_name='/services/orders',
    create_log_group=False,
)

Pre-provisioning is the clearer way to own retention, encryption, tags, and IAM; startup fails to deliver if the group is absent.

Add Watchtower through Django logging settingsconfigure-django

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'cloudwatch': {
            'class': 'watchtower.CloudWatchLogHandler',
            'level': 'INFO',
            'log_group_name': 'my-django-service',
        },
    },
    'root': {'handlers': ['cloudwatch'], 'level': 'INFO'},
}

The README disables propagation for some Django development-server loggers because handler threading can deadlock there.

Flush queued logs during orderly shutdownflush-on-shutdown

try:
    run_worker()
finally:
    handler.flush()
    handler.close()
    logger.removeHandler(handler)

Orderly flushing reduces loss, but a hard kill or frozen runtime can still strand the asynchronous queue.

Keep AWS SDK debug noise off the cloud handleravoid-sdk-recursion

for name in ('boto3', 'botocore', 'urllib3'):
    logging.getLogger(name).setLevel(logging.WARNING)

logger.addHandler(handler)

Watchtower filters these libraries on its own handler to prevent recursive sends, but other attached handlers still receive their records.

Alternatives

Package	Registry	Pick it when
python-json-logger	PyPI	Your runtime already forwards stdout and only needs structured JSON from standard logging
structlog	PyPI	You want structured event processing and can leave cloud transport to the platform collector
loguru	PyPI	A smaller application values easy sinks and readable local logs more than standard logging handler compatibility