PostHog: analytics on billions of events
Product analytics that outgrew Postgres: an event pipeline into a column store, and the copying that made person queries fast.
The idea
Product analytics asks questions row databases are bad at: count events matching a filter over months, grouped by a property, for one customer among thousands. PostHog started on Postgres and moved to ClickHouse, a column store that reads only the columns a query touches and compresses them well.
Their posts are useful because they cover the costs, not only the speed-up. Events arrive through Kafka rather than direct inserts, so spikes and database outages are absorbed. Changing existing rows in a column store is expensive, and doing it often caused real trouble. Properties stored as one JSON blob had to be parsed on every query until the most used ones were given columns of their own. And joining events to people at query time was slow enough that they now copy person data onto each event as it arrives, accepting that a person's later changes no longer rewrite history.
Read the originals
Written by the engineers who built it.
- How we're improving performance by combining persons and events
PostHog · Post, Nov 2022
Copying person data onto every event to avoid a join at query time, and what that changes about how merged users appear in old data.
- How we turned ClickHouse into our event mansion
James Greenhill · Post, Nov 2021
Why event analytics outgrew Postgres, how they chose a column store, and the mistakes they made running it.
- How to speed up ClickHouse queries using materialized columns
Karl-Aksel Puulmann · Post, Oct 2021
Finding out where query time goes (parsing JSON), and pulling hot properties into their own columns without rewriting the table.
Practise it
Make the decisions yourself, then compare.