Skip to content
sysgeeks

Design a News Feed (Twitter Timeline), stage 8 of 9: change it

Top posts first

Rankings depend on the reader, the post, and signals such as replies and likes that arrive over the following minutes and hours.

System so far· 9 parts
123456789CLIENTAuthorsSERVICEWrite APIDATABASEPost storeQUEUEFan-out queueWORKERFan-out workersSERVICESocial graphCACHETimeline cacheSERVICETimeline serviceCLIENTReaders

Select a component to see what it is responsible for and which state it owns.

  1. 1Authors → Write API: Post
  2. 2Write API → Post store: Store post
  3. 3Write API → Fan-out queue: Fan-out job
  4. 4Fan-out workers → Fan-out queue: Take jobs
  5. 5Fan-out workers → Social graph: Active followers
  6. 6Fan-out workers → Timeline cache: Push ID, trim to 800
  7. 7Readers → Timeline service: GET home timeline
  8. 8Timeline service → Timeline cache: Page of IDs
  9. 9Timeline service → Post store: Hydrate; large accounts' recent posts
  • Request / response
  • Asynchronous

What you need to know

0 of 2 checks done

  1. A ranked feed has two steps:

    1. Candidate generation: what could this reader see? (Posts from accounts they follow, maybe more.)
    2. Ranking: in what order? (Scored using the reader, the post, and engagement signals.)

    The first changes only when posts or follows change. The second changes every minute as likes and replies arrive.

  2. Think first

    Scores are computed at fan-out time and stored with each timeline entry. A post gets most of its likes in the hour after it's published. What's wrong?