Kafka is a shared log for systems that cannot move in lockstep
Apache Kafka accepts records from producers, retains them in ordered partitions, and lets consumers read at their own pace. That model works for service events, data integration, and streaming applications because a producer does not need a direct connection to every downstream system. Consumers can also replay retained records when they rebuild state or recover from an outage.
The repository is large enough to demand deliberate ownership. Our checkout contained 7,495 files, about 1,599,276 lines of source, and occupied 86 MB before the measured dependency install. The top-level project includes the broker, clients, Kafka Streams, tests, examples, build tooling, and release machinery. A team adopting Kafka is taking on a platform, not adding a small queue library to one application.
Ordering is scoped by partition, which is central to design work. More partitions create more parallel work, while related events often need the same partition if their order matters. Retention also separates Kafka from a simple handoff queue: a consumer's successful read does not inherently erase the record. Those choices help replay and fan-out, but they require teams to define keys, schemas, retention, and failure behavior before production traffic arrives.
What happened when we ran it
Our sandbox cloned commit 191a7fb with 3 CPUs and 8 GB of RAM, then worked inside ./committer-tools, the Python project selected by the harness. Installation succeeded in 67 seconds, adding 45 packages and using 63 MB on disk. Its build succeeded in 15 seconds. Pip-audit reported 0 known vulnerabilities in that installed Python environment.
The test step failed with exit code 5 after 9 seconds. Pytest printed no tests ran in 0.00s, which means it collected 0 tests and therefore reported 0 passed and 0 failed of 0. There was no failing assertion or broker error in the supplied log. The finding is simply that this command did not execute a test suite.
That scope matters more than the green build. We did not build the Java and Scala broker, run ./gradlew test, format broker storage, start a server, or send records. The repository has 20 CI workflow files and a tests directory, but those signals do not turn our Python subproject run into broker evidence. Any claim about Kafka throughput, durability, or full-suite status would need another run.
Java 17 is the contributor baseline
The README says Kafka is built and tested with Java 17 and 25. Client and Streams modules target Java 11 compatibility, while the rest targets Java 17. Scala 2.13 is the only supported Scala version. Developers using IntelliJ are told to use JDK 17, and Gradle tasks cover JARs, documentation, unit tests, integration tests, coverage, formatting, static analysis, and release archives.
A local broker can be started after generating a cluster ID and formatting storage with the supplied server configuration. The README also gives a one-line Docker start on port 9092. Both are useful experiments. Neither covers a production cluster's storage capacity, replication policy, listener security, authorization, monitoring, recovery testing, or upgrade sequence.
The build instructions are unusually frank about test controls. Gradle can run unit and integration suites separately, include tests marked flaky, retry a limited number of failures, and rerun one case hundreds of times. The repository even links a flaky-test report from its README. That is the kind of machinery expected in a mature infrastructure project, although our 9-second pytest attempt never exercised it.
Operating the broker is harder than starting port 9092
Kafka can remove direct dependencies between many producers and consumers, but it creates a new dependency on the cluster. Operators must know when disks are filling, replicas are unhealthy, consumers are falling behind, or partition choices are limiting a workload. Authentication and authorization also need to be designed around the clients that publish and read sensitive events.
The 86 MB checkout and 1.59 million source lines give a rough sense of the code surface, not runtime cost. We measured no broker memory, disk throughput, request latency, or recovery time. Capacity advice copied from another company would be especially weak here because event size, retention, replication, producer behavior, and consumer lag change the answer. Benchmark the actual record shape and failure cases on the target infrastructure.
Kafka is also a poor default when the requirement is merely delayed work for a few processes. A simpler queue can offer acknowledgements and retries without forcing the team to reason about partition counts and retained logs. Kafka earns its operational bill when replay, several independent consumers, sustained event history, or the existing Kafka ecosystem are genuine requirements.
Active development does not remove the staffing requirement
GitHub recorded the last push on August 26, 2026, and listed 33,628 stars with 524 open issues and pull requests. The latest-release API returned no GitHub release, so release tags alone are not a useful health signal for this mirror. Current push activity and the visible issue and pull-request queue show ongoing work, while the combined open count should not be mistaken for 524 confirmed bugs.
Kafka deserves a shortlist for a company-wide event backbone because the protocol, clients, Streams API, and operating knowledge are widely available. It does not deserve automatic selection for every asynchronous job. The deciding evidence should come from the proposed event contracts, replay needs, failure model, and a workload test on hardware the team can actually operate.