Design a Product Analytics System, stage 6 of 9: break it
Two visitors who were one person
Anonymous events carry a random device ID. At signup, the snippet sends an identify call linking that device ID to the new account. Person data is copied onto events at ingestion, as decided in the last stage.
System so far· 8 parts
Select a component to see what it is responsible for and which state it owns.
- 1Customers' apps → Capture API: Batches of events
- 2Customers' dashboards → Query service: Chart request
- 3Query service → Postgres: Teams and saved charts
- 4Query service → Events (column store): Aggregate by column
- 5Capture API → Event stream: Append, then acknowledge
- 6Ingestion workers → Event stream: Read a partition
- 7Ingestion workers → Postgres: Who is this ID?
- 8Ingestion workers → Events (column store): Batch insert
What you need to know
0 of 2 checks done
Before signup, a visitor's events carry a random device ID. At signup, an identify call links that device ID to the new account. But earlier events were already stored with the device ID, and in a column store nothing rewrites stored events cheaply.
So after the link, the same human appears as two people in older data.
Check
What's a cheap way to make funnels count them as one person without rewriting events?Think first
The identify call and the user's next event are processed by different workers, and the event is processed first. What goes wrong?