PyPIUtilsupdated 06 Aug 2026

fastavro

Item: fastavro
Rating: 3.8
Author: Keyoor

fastavro reads and writes Avro data from Python using Cython C extensions instead of the pure-Python code in Apache's official avro package. Same file format, same schema rules, roughly an order of magnitude faster on the read path (the README cites 1.7 seconds versus 14 for the same 10,000 record file). It covers the container file format with deflate, snappy, zstandard, bzip2, lz4 and xz codecs, the schemaless single-record encoding that Kafka pipelines use, Avro's JSON encoding, schema resolution and aliases, logical types like timestamps and decimals, and canonical-form fingerprinting. The one thing it deliberately does not implement is Avro's RPC layer.

fastavro · v1.12.2 · 10.7M/wk · ★ 711 · pushed 25 Jul 2026 · fastavro/fastavro · docs · reviewed by Keyoor

Verdict

Still the fastest and most complete Avro implementation in Python, and the one nearly every Kafka Python stack ends up depending on whether or not you chose it. Install it with clear eyes: the README says maintenance mode, so it will keep working but missing features are permanent.

API stability5/5The 1.x line has been current for years with additive minor releases; writer, reader, parse_schema and the schemaless pair have not changed shape, and maintenance mode makes future breakage even less likely than usual.

Docs4/5fastavro.readthedocs.io documents every public function with runnable examples and has dedicated pages for logical types and schema resolution; what is missing is judgement, such as which codec to pick or how to model unions, which you end up learning from GitHub issues.

Maintenance2/5The README declares maintenance mode and warns that even simple bug fixes may be significantly delayed. Releases do still ship (1.12.2 in April 2026, repo pushed July 2026) but 54 open issues sit against one maintainer who has capped scope on purpose.

Ecosystem4/5About 10.7M downloads a week, largely transitive through Kafka clients and data tooling, and it is available on conda-forge. Stack Overflow coverage is good for read/write basics and thin for schema evolution corners.

Use it if

You are pushing records through Kafka or Pulsar and schemaless_writer/schemaless_reader sit in your hot loop; this is why most people install it, often transitively through confluent-kafka's Avro serializers
You already have .avsc schemas produced by a JVM service and need Python consumers that agree on schema resolution, aliases, defaults and logical types rather than approximating them
You need to read Avro container files written by Spark, Hadoop or Flink from a plain Python process with no JVM and no Arrow dependency
You want to_parsing_canonical_form and fingerprint so you can key a schema registry cache by schema identity instead of by raw JSON text

Skip it if

You need active feature development: the README opens with a maintenance-mode notice saying the maintainer will try to keep up with Python versions, security issues and simple bug fixes, and that even those might be significantly delayed. Treat the current feature set as the final one
You want Avro RPC or protocol support: the README lists it under Missing Features and there is no plan to add it
Avro is not a hard requirement and your workload is analytical: Avro is row-oriented, so scanning three columns out of eighty costs you the whole row. Parquet through pyarrow gives column pruning and predicate pushdown for the same data
You want typed records: the API is dicts in, dicts out. There is no dataclass or pydantic codegen, so the mapping between your domain objects and Avro records is code you write and maintain yourself
Your deployment target has no matching wheel: the speed comes from compiled extensions, and when the build fails pip can still install a pure-Python fallback that performs like the reference implementation you were trying to escape
Bus factor matters: 54 open issues (66 counting PRs) against a project run by a single maintainer who has publicly capped his own commitment

Setup reality

pip install fastavro gets you the container format plus deflate, bzip2 and xz, because those codecs come from the standard library. Everything else is an extra: fastavro[snappy] and fastavro[zstandard] both pull cramjam or backports.zstd rather than the old python-snappy that needed libsnappy headers, and lz4 is its own extra again. Wheels exist for CPython on the usual platforms so most people never see a compiler, but when no wheel matches, pip builds the Cython extension from source and, if that fails, quietly installs the pure-Python implementation. Nothing warns you at import time, so if throughput suddenly looks like the reference library, check whether fastavro.schema._schema.__file__ ends in .so or in .py. The other recurring surprise is parse_schema: it rewrites your schema dict and stamps a __fastavro_parsed hint into it, so do not hand the parsed object to another Avro library or compare it against your original JSON.

Patterns

Write records to an Avro container filewrite-avro-file

from fastavro import writer, parse_schema

schema = {
    "type": "record",
    "name": "Weather",
    "namespace": "test",
    "fields": [
        {"name": "station", "type": "string"},
        {"name": "time", "type": "long"},
        {"name": "temp", "type": "int"},
    ],
}
parsed = parse_schema(schema)

records = [
    {"station": "011990-99999", "time": 1433269388, "temp": 0},
    {"station": "011990-99999", "time": 1433270389, "temp": 22},
]

with open("weather.avro", "wb") as out:
    writer(out, parsed, records)

records can be any iterable including a generator, so you never need the whole dataset in memory. Open the file in binary mode; text mode raises a confusing encoding error deep inside the encoder.

Iterate an Avro file and read its metadataread-avro-file

from fastavro import reader

with open("weather.avro", "rb") as fo:
    avro_reader = reader(fo)
    print(avro_reader.writer_schema["name"])
    print(avro_reader.codec)      # 'null', 'deflate', 'snappy', ...
    print(avro_reader.metadata)   # header key/value pairs

    for record in avro_reader:
        process(record)

Use .writer_schema, not .schema: the plain .schema attribute is deprecated and emits a DeprecationWarning. The reader is a one-shot iterator, so seek back to 0 and build a new reader if you need a second pass.

Encode a single record for Kafka (no container header)schemaless-encode-decode

from io import BytesIO
from fastavro import schemaless_writer, schemaless_reader, parse_schema

parsed = parse_schema(schema)

buf = BytesIO()
schemaless_writer(buf, parsed, {"station": "a", "time": 1, "temp": 3})
payload = buf.getvalue()      # send this as the Kafka message value

record = schemaless_reader(BytesIO(payload), parsed)

Schemaless bytes carry no schema, so the consumer must already know it; that is what a schema registry is for. schemaless_reader takes the writer schema first and an optional reader schema second, and getting that order wrong decodes garbage rather than raising.

Read old files with a new schemaevolve-schema-on-read

from fastavro import reader

reader_schema = {
    "type": "record",
    "name": "Weather",
    "namespace": "test",
    "fields": [
        {"name": "station", "type": "string", "aliases": ["station_id"]},
        {"name": "time", "type": "long"},
        {"name": "temp", "type": "int"},
        {"name": "humidity", "type": ["null", "int"], "default": None},
    ],
}

with open("old-weather.avro", "rb") as fo:
    for record in reader(fo, reader_schema):
        print(record["humidity"])  # None for old records

A new field only resolves if it has a default; without one, fastavro raises a SchemaResolutionError instead of filling in null. Aliases let you rename a field without rewriting history.

Append records to an existing Avro fileappend-to-avro-file

from fastavro import writer, parse_schema

parsed = parse_schema(schema)
more = [{"station": "012650-99999", "time": 1433275478, "temp": 111}]

with open("weather.avro", "a+b") as out:
    writer(out, parsed, more)

The a+b mode matters: fastavro reads the existing header to reuse the sync marker and codec. Opening with 'ab' alone is not readable and fails, and appending with a different schema raises rather than silently corrupting the file.

Load .avsc files, including cross-file referencesload-schema-from-file

from fastavro.schema import load_schema, load_schema_ordered

# resolves named types found in sibling .avsc files in the same directory
parsed = load_schema("schemas/Order.avsc")

# when the files are not co-located, give the dependency order yourself
parsed = load_schema_ordered(
    ["schemas/Address.avsc", "schemas/Customer.avsc", "schemas/Order.avsc"]
)

load_schema only auto-discovers named types in the same folder as the entry file. If your schemas live in nested directories, load_schema_ordered with an explicit dependency-first list is the only reliable route.

Disambiguate union branches with tuple notationwrite-union-values

schema = {
    "type": "record",
    "name": "Event",
    "fields": [
        {
            "name": "payload",
            "type": [
                {"type": "record", "name": "Click", "fields": [{"name": "url", "type": "string"}]},
                {"type": "record", "name": "View", "fields": [{"name": "url", "type": "string"}]},
            ],
        }
    ],
}

record = {"payload": ("Click", {"url": "/pricing"})}  # branch name, then value

Two record branches with identical field sets are ambiguous, and fastavro picks the first match, which is often wrong. The (name, value) tuple forces the branch. On the way back, pass return_record_name=True to reader so you can tell which branch you got.

Round-trip timestamps and decimalsuse-logical-types

from datetime import datetime, timezone
from decimal import Decimal

schema = {
    "type": "record",
    "name": "Payment",
    "fields": [
        {"name": "at", "type": {"type": "long", "logicalType": "timestamp-micros"}},
        {
            "name": "amount",
            "type": {"type": "bytes", "logicalType": "decimal", "precision": 12, "scale": 2},
        },
    ],
}

record = {"at": datetime.now(timezone.utc), "amount": Decimal("19.99")}

Naive datetimes are assumed to be UTC on write and come back timezone-aware, so a naive value written in local time shifts. A Decimal with more digits than the declared scale raises rather than rounding.

Validate records against a schemavalidate-before-writing

from fastavro.validation import validate, validate_many
from fastavro import parse_schema

parsed = parse_schema(schema)

validate({"station": "a", "time": 1, "temp": 3}, parsed)          # True or raises
validate_many(records, parsed)

# collect every problem instead of stopping at the first
validate(bad_record, parsed, raise_errors=False)  # returns False

Validation is a separate pass over the data, so running it on every record in a high-throughput producer roughly doubles the cost. Validate in tests and on untrusted input, not in the hot path.

Fingerprint a schema for registry cachingfingerprint-schema

from fastavro.schema import to_parsing_canonical_form, fingerprint

canonical = to_parsing_canonical_form(schema)
print(fingerprint(canonical, "CRC-64-AVRO"))
print(fingerprint(canonical, "sha256"))

fingerprint expects the canonical form, not the raw schema; hashing raw JSON gives different results for semantically identical schemas because of key order and whitespace. CRC-64-AVRO is the algorithm the Avro spec defines for single-object encoding.

Read a large file block by blockread-in-blocks

from fastavro import block_reader

with open("large.avro", "rb") as fo:
    for block in block_reader(fo):
        print(block.num_records, block.offset, block.size)
        for record in block:
            process(record)

Blocks are the natural split boundary for parallel or distributed reads, since each one is independently decodable given the header. Iterating a block consumes it, so capture num_records before the loop if you need the count.

Generate fake records that match a schemagenerate-test-records

from fastavro.utils import generate_many, anonymize_schema

for record in generate_many(schema, 100):
    producer.send(record)

# strip field names before sharing a schema in a bug report
safe = anonymize_schema(schema)

Generated values are random and meaningless, which is what you want for throughput benchmarks and what you do not want for assertion-based tests. It respects unions and logical types, so it is a fast way to prove a schema actually encodes.

Alternatives

Package	Registry	Pick it when
avro	PyPI	You need the Apache reference implementation's exact behaviour or its RPC support, and throughput is not a concern.
pyarrow	PyPI	You control the storage format and your access pattern is analytical, so columnar Parquet beats row-oriented Avro.
confluent-kafka	PyPI	You are on Confluent Schema Registry and want registry-aware serializers instead of wiring schema IDs to schemaless encode calls yourself.