mrkeyoor.com_
Tue 03 Nov 02:49 UTC
Tech3 min read

PostHog rebuilds its warehouse on Trino after DuckDB sizing problems

PostHog says unpredictable query demands strained its DuckDB deployment. Its Trino rebuild changes how work is allocated, while comparative costs remain unproven.

PostHog said on November 2 that it has rebuilt its warehouse around Trino and its own table format, HogLake, after struggling to provision DuckDB workers for unpredictable customer queries. The account gives warehouse operators a specific problem to examine: how to allocate resources before knowing what customers will ask.

PostHog reports that complex queries required larger instances. It supplies no reproducible comparison establishing the replacement's performance or cost, and MrKeyoor hasn't tested either deployment.

What distributed execution changes

Trino's architecture documentation describes a coordinator that plans queries and assigns tasks to workers across a cluster. Workers fetch source data and exchange intermediate results. The coordinator collects the final results for the client. A query can therefore use multiple servers, with coordination and network transfers part of the execution path.

For an operator, that execution model creates a different sizing question. Instead of only asking how large one worker must be, an evaluation needs to consider task distribution and data exchange between workers. Distributed execution offers another way to allocate work. It still needs testing against the query mix and costs of the actual service, rather than an assumption that more machines mean cheaper queries.

Query execution wasn't the only problem

PostHog's account also describes DuckLake catalog overhead: commits reread statistics for the entire catalog. It separately reports difficulty supporting the range of behaviors expected by PostgreSQL clients. Those problems concern metadata and client compatibility, which should be assessed separately from query execution speed.

PostHog says it tested the replacement internally before handing it to design partners. Watch for follow-up evidence on customer workloads and operating costs before treating its rebuild as a reason to migrate.

Sources

  1. Why we're moving our warehouse off DuckDB
  2. Trino concepts: architecture and query execution