boto3
Boto3 is the official AWS SDK for Python, maintained and published by Amazon. It gives you a Python client for essentially every AWS service (S3, EC2, DynamoDB, SQS, Lambda and hundreds more), generated from AWS's own service definitions via the underlying botocore library. It handles credential resolution, request signing, retries, pagination and waiters. If you touch AWS from Python, whether in a script, a server or inside Lambda, this is the tool; there is no serious competitor for direct AWS access.
Mandatory and dependable if you use AWS from Python; the API is stable and Amazon ships releases almost daily. Just budget for the credentials learning curve, add boto3-stubs for sane autocomplete, and prefer clients over the frozen resource API.
Use it if
- You automate or integrate anything on AWS from Python: it is the official, always-current SDK with same-day support for new services
- You run code in AWS Lambda, where boto3 is preinstalled in the Python runtime and picks up the execution role's credentials automatically
- You need production plumbing done right: built-in paginators, waiters, configurable retry modes and request signing you should never hand-roll
- You want AWS-backed support and a maintenance policy rather than a community wrapper
- You need async: boto3 is synchronous and blocks the event loop; aioboto3 exists but is a community wrapper, not AWS-supported
- You mostly move dataframes around: awswrangler (AWS SDK for pandas) wraps boto3 with far less ceremony for S3/Athena/Redshift analytics work
- You want typed autocomplete out of the box: clients are generated at runtime, so your editor sees nothing until you add third-party boto3-stubs
- You are not on AWS or need multi-cloud abstraction; this SDK is AWS-only by design
Setup reality
pip install boto3 works, but the install is heavy: botocore ships JSON service definitions for every AWS service, so expect tens of megabytes, which matters for slim containers and Lambda layers. The real setup work is credentials: the resolution chain (env vars, ~/.aws/credentials, ~/.aws/config profiles, SSO, instance/task roles) is powerful and a classic source of it-works-on-my-machine bugs. Editor autocomplete requires installing boto3-stubs with the right service extras because the client factory is dynamic. Python 3.9 support ended in April 2026; you need 3.10+. Also note the resource API (boto3.resource) is feature-frozen; new work should use clients.
Patterns
Create a client with explicit regionclient-setup
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
# or from a named profile
session = boto3.Session(profile_name="prod", region_name="eu-west-1")
s3 = session.client("s3")Never hardcode keys in code; let the credential chain (env vars, profiles, IAM roles) supply them. Clients are thread-safe, sessions are not.
Upload and download S3 filess3-upload-download
import boto3
s3 = boto3.client("s3")
s3.upload_file("report.pdf", "my-bucket", "reports/report.pdf")
s3.download_file("my-bucket", "reports/report.pdf", "/tmp/report.pdf")upload_file/download_file handle multipart transfers and retries automatically; prefer them over raw put_object for anything larger than a few MB.
Generate a presigned URLs3-presigned-url
url = s3.generate_presigned_url(
"get_object",
Params={"Bucket": "my-bucket", "Key": "reports/report.pdf"},
ExpiresIn=3600,
)The URL is signed with the caller's credentials; if those are temporary (STS/role), the link dies when the credentials expire, whatever ExpiresIn says.
List all objects with a paginatorpagination
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", Prefix="logs/"):
for obj in page.get("Contents", []):
print(obj["Key"], obj["Size"])list_objects_v2 caps at 1000 keys per call; skipping the paginator and missing that cap is a classic silent-truncation bug.
Handle AWS errors by error codeerror-handling
from botocore.exceptions import ClientError
try:
s3.head_object(Bucket="my-bucket", Key="maybe-missing")
except ClientError as e:
code = e.response["Error"]["Code"]
if code == "404":
print("not found")
else:
raiseAlmost every service error is a ClientError; branch on the code string, not the exception type. head_object returns 404, get_object returns NoSuchKey.
Configure retries and timeoutsretry-config
from botocore.config import Config
cfg = Config(
retries={"max_attempts": 10, "mode": "adaptive"},
connect_timeout=5,
read_timeout=60,
)
dynamodb = boto3.client("dynamodb", config=cfg)Default is legacy retry mode; adaptive adds client-side rate limiting, which is what you want when you keep hitting Throttling errors.
DynamoDB put and getdynamodb-crud
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("users")
table.put_item(Item={"pk": "user#1", "name": "Ada", "score": 99})
resp = table.get_item(Key={"pk": "user#1"})
item = resp.get("Item")The resource API is worth keeping for DynamoDB specifically because it converts Python types for you; the raw client makes you write {"S": "..."} type descriptors.
Assume an IAM role for cross-account accessassume-role
sts = boto3.client("sts")
creds = sts.assume_role(
RoleArn="arn:aws:iam::123456789012:role/Deploy",
RoleSessionName="deploy-script",
)["Credentials"]
s3 = boto3.client(
"s3",
aws_access_key_id=creds["AccessKeyId"],
aws_secret_access_key=creds["SecretAccessKey"],
aws_session_token=creds["SessionToken"],
)Temporary credentials expire (default 1 hour) and boto3 will not auto-refresh ones you pass manually; long-running jobs should re-assume or use a profile with role_arn.
Send and receive SQS messagessqs-send-receive
sqs = boto3.client("sqs")
queue_url = sqs.get_queue_url(QueueName="jobs")["QueueUrl"]
sqs.send_message(QueueUrl=queue_url, MessageBody='{"job": 1}')
msgs = sqs.receive_message(
QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=20
)
for m in msgs.get("Messages", []):
process(m["Body"])
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=m["ReceiptHandle"])You must delete messages explicitly after processing or they reappear after the visibility timeout; WaitTimeSeconds=20 (long polling) cuts cost and empty responses.
Wait for a resource to reach a statewaiters
ec2 = boto3.client("ec2")
resp = ec2.run_instances(ImageId="ami-123", InstanceType="t3.micro",
MinCount=1, MaxCount=1)
instance_id = resp["Instances"][0]["InstanceId"]
waiter = ec2.get_waiter("instance_running")
waiter.wait(InstanceIds=[instance_id])Waiters poll with sane backoff and give up after a service-specific limit; tune via waiter.wait(..., WaiterConfig={"Delay": 15, "MaxAttempts": 40}).
Read a large S3 object without loading it allstreaming-body
resp = s3.get_object(Bucket="my-bucket", Key="big.csv")
for line in resp["Body"].iter_lines():
handle(line)resp["Body"] is a streaming object; calling .read() with no args pulls the whole file into memory, which is how Lambdas run out of RAM.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| aioboto3 | PyPI | You need async AWS calls inside asyncio applications; community-maintained wrapper over aiobotocore |
| awswrangler | PyPI | Pandas-centric data work against S3, Athena, Glue or Redshift with much less boilerplate |
| s3fs | PyPI | You just want S3 to look like a filesystem for pandas, dask or fsspec-aware tools |