Cloudflare's second reported 100TB memory reduction in less than a month began with a load-balancing process that sometimes consumed 6GB. The culprit was not a leak. Its Pingora Backend Router held far more points on its consistent-hash rings than the traffic distribution needed, and each point occupied eight bytes in Rust. Cloudflare packed those records into six bytes and cut the points generated for each server by 90%. It then reported a fleet-wide memory drop of 100TB in its engineering account.
The post had reached 333 Hacker News points when MrKeyoor's news brief captured it. That is a measure of developer attention, not independent proof of Cloudflare's result. The useful part for other teams is testable: a conservative default can become an enormous allocation after capacity weights and feature combinations multiply it. The Hacker News discussion also contains proposed alternatives to ketama hashing, but those suggestions do not come with comparable production measurements.
Why a ring ate gigabytes
Pingora Backend Router, or PBR, sends cacheable requests to servers by URL. Cloudflare says this lets a data center keep one copy of a cached file and find that copy again. Consistent hashing gives servers and requests positions in a fixed integer space, then assigns each request to a server at the next relevant position. When a server joins or leaves, most other assignments can remain where they were instead of being remapped across the whole pool, as the company's PBR explanation describes.
One position per server distributes work badly because hash outputs are random. In Cloudflare's 100-server example, each server should receive 1% of the range, yet the coefficient of variation is about 99%. Some servers can therefore receive close to twice their intended share while others get very little. NGINX addresses this by using 160 points per server, and Pingora inherited the same default. Cloudflare's calculation puts the variation near 8% at 160 points, according to the consistent-hashing analysis.
PBR also has to account for unequal storage. A server with more disk should accept more cacheable traffic, so its weight increases its number of hash points. Cloudflare illustrates the multiplication with a weight of 625: the 160-point base becomes 100,000 points for one server. Compliance rules and enabled cache features then restrict which machines may handle a request. PBR needs separate rings for the possible feature combinations, producing dozens of rings rather than one. The resulting structures pushed some processes to 6GB.
The scale error sat between components. A default chosen to smooth distribution was multiplied by disk capacity, then copied into each eligible-server combination. Looking only at the eight-byte record or the 160-point default would miss most of the bill. Cloudflare found the problem after a performance ticket traced excessive PBR memory to pingora-ketama.
Two bytes mattered because the count was wrong
Each ring point originally stored a 32-bit hash and a 32-bit server index in the reported implementation:
struct Point {
hash: u32,
index: u32,
}
That record occupies eight bytes. PBR is unlikely to coordinate more than 65,535 servers at once, so a 16-bit index is enough. Simply changing index to u16 still leaves an eight-byte record because Rust aligns the structure to the four-byte boundary required by u32. Cloudflare instead stored the fields in a six-byte array and exposed getters for the two values, as its storage account shows:
struct Point([u8; 6]);
The packed representation reduced the memory used for consistent hashing by 25%. The authors avoided #[repr(packed)], whose unaligned fields can make references unsafe or awkward, and report that the byte-array version compiled to the same machine operations in their comparison. This Rust change explains only part of the 100TB total. The larger reduction came from proving that PBR did not need most of the records, according to Cloudflare's breakdown.
Cloudflare derived the variation for multiple points per server and found sharply diminishing returns. In its worked case, raising a server from 10,000 to 100,000 points improved the estimated error by only 0.7%. A second limit appears because PBR uses 32-bit hashes. At high point counts, collisions stop the ring from behaving like a continuous mathematical space. The company's simulation for a 2,048-server data center shows the error beginning to rise somewhere between 10,000 and 100,000 points per server.
Those results gave the team room to generate 90% fewer points without what it described as an appreciable loss of distribution accuracy. The 100TB figure combines that reduction with the denser record and related implementation work. It is a company-reported production measurement supported by the post's math and internal memory graphs. The post does not provide a raw fleet dataset or an outside reproduction, so its percentage and aggregate should be read as Cloudflare's result for PBR rather than a general ketama benchmark.
The dangerous line was the migration switch
Changing a consistent-hash ring changes which backend receives some URLs. A fleet-wide switch would have made much of Cloudflare's cached content unreachable at its previous location. Origins could then receive a sudden wave of requests even if the new ring distributed traffic perfectly. Cloudflare's rollout account treats that cache churn as the main operational risk.
For a period, PBR kept both ring versions in memory. A stable decision derived from each request hash chose the old or new backend selector, which also allowed Cloudflare to move requests back without redeploying PBR. The rollout began in small validation locations and advanced through larger groups of data centers. Cloudflare separately controlled how much traffic used the new ring and where movement was allowed, keeping cache churn within chosen locations instead of spreading it across the network, according to its migration description.
The team watched backend-selection traces and ring-version counters alongside PBR connection errors, process memory, startup time, cache behavior and origin traffic. Once every location had moved to the new ring, Cloudflare removed the old path. Its published graph shows the sharp memory drop on the day the unused large rings were decommissioned. Memory falls only after the compatibility copy is gone, while safety depends on keeping the old decision available during the move.
The new path is open source and opt-in
Cloudflare has put the changes in pingora-ketama, the Apache 2.0-licensed Rust port of NGINX's consistent-hash function. The repository's current Cargo manifest identifies version 0.9.0 and exposes v2 as an optional feature backed by i_key_sort. Cloudflare says that path has the six-byte storage format and a configurable base point count.
The feature remains deliberately quiet. Version 1 preserves the existing ring, and the library can run both versions at once so an application can choose per request. Installing the crate alone does not deliver Cloudflare's memory result. A deployment must opt in, choose a lower point count suited to its server weights, and measure distribution and cache movement under its own traffic, as the release explanation makes plain.
This is Cloudflare's second reported 100TB reduction since August 27, but the earlier work attacked a different multiplier. The company's 1.1.1.1 platform holds more than 250 billion DNS cache entries, where one wasted byte across every entry costs over 250GB. Five storage changes cut the measured entry footprint from 953 to 420 bytes and reduced fleet memory by roughly 100TB, according to the earlier DNS cache report. The new PBR work removed excess objects from duplicated hash rings. Together, the cases point to the same audit question: which small allocation is being repeated far more often than its original author expected?
Outside deployments are the next useful evidence. Watch whether v2 becomes an advertised default and whether Pingora users publish before-and-after measurements across varied server weights and feature combinations. Cloudflare's dual-ring rollout gives those users a practical way to test the change without betting their origin capacity on one switch. The six-byte record is easy to copy. The harder proof is a stable cache after most of the ring has disappeared.