Skip to content
sysgeeks

Design a News Feed (Twitter Timeline), stage 5 of 9: decide

Timelines nobody reads

Many accounts have not opened the app in months, yet every post from someone they follow is still inserted into their timeline. Twitter stopped fanning out to users who had not logged in for 30 days.

System so far· 9 parts
123456789CLIENTAuthorsSERVICEWrite APIDATABASEPost storeQUEUEFan-out queueWORKERFan-out workersSERVICESocial graphCACHETimeline cacheSERVICETimeline serviceCLIENTReaders

Select a component to see what it is responsible for and which state it owns.

  1. 1Authors → Write API: Post
  2. 2Write API → Post store: Store post
  3. 3Write API → Fan-out queue: Fan-out job
  4. 4Fan-out workers → Fan-out queue: Take jobs
  5. 5Fan-out workers → Social graph: Active followers
  6. 6Fan-out workers → Timeline cache: Push ID, trim to 800
  7. 7Readers → Timeline service: GET home timeline
  8. 8Timeline service → Timeline cache: Page of IDs
  9. 9Timeline service → Post store: Hydrate; large accounts' recent posts
  • Request / response
  • Asynchronous

What you need to know

0 of 2 checks done

  1. Derived data is computed from other data and can be rebuilt from it. A timeline is derived: it's a function of the post store and the follow graph.

    Anything derived can be skipped, evicted or lost without losing anything durable. The cost is the work to rebuild it when it's needed again. See Soft state.

  2. Check

    Fan-out skips users inactive for 30 days. One of them returns. What do they see?