Publish/subscribe
Decoupling senders from receivers by topic: a publisher sends once and every current subscriber receives a copy.
Communication
Learn it
Several processes need to react to the same event: every server holding a connection for a chat room, every service that cares that an order was paid. If the producer calls each one, it's coupled to all of them and to their availability.
Publish/subscribe decouples them: publishers send to a topic, subscribers register interest, and the broker delivers a copy to each. The publisher doesn't know who receives it.
Check
A new analytics service wants to know when orders are paid. With pub/sub, what changes in the order service?The crucial difference between systems is what happens to subscribers who aren't listening:
- Ephemeral pub/sub (Redis PUBLISH/SUBSCRIBE, most in-memory buses): if you're disconnected when a message is published, you never get it. Good for live fan-out where a miss is recovered another way (a reload, a resync from a log).
- Durable pub/sub (Kafka consumer groups, SNS → SQS, Google Pub/Sub): each subscription keeps its own backlog, so a subscriber that was down catches up. You pay in storage and per-subscriber position tracking.
Think first
Collaborative-editor servers use ephemeral pub/sub to forward each document's operations. A server misses a message during a 2-second network blip. How does it recover?Pub/sub fans out messages. It doesn't order them across publishers, and it doesn't make subscribers' handling idempotent. Those remain the receivers' problems.
Quick reference
The same ideas, condensed for revision.
How it goes wrong
- Missed while disconnected
- With ephemeral pub/sub, a subscriber that reconnects has a gap it does not know about.
- Slow subscriber
- One subscriber falls behind; depending on the broker it is dropped, buffered without limit, or slows the publisher.
- Hot topic
- A single topic with enormous fan-out concentrates load on one broker node.
Instead, consider
- Direct calls
- There is exactly one consumer and you need its answer.
- Routing all participants to one process
- Fan-out is within a small group, such as one document's editors, and one owner can deliver locally.
- Work queue
- Each message should be handled by exactly one of several workers, not by all of them.
In practice
- Redis Pub/Sub
- Fire-and-forget fan-out between servers; no retention.
- Postgres LISTEN/NOTIFY
- Lightweight notifications tied to transactions; no retention, small payloads.
- SNS → SQS, Google Pub/Sub
- Durable per-subscriber delivery.
- Kafka topics with consumer groups
- Retained log; each group tracks its own offset.
It assumes
- Subscribers either tolerate missing messages (ephemeral) or the broker retains them per subscriber (durable).
- The set of topics and their fan-out is bounded enough for the broker to handle.
Explain it in your own words
Where you practise it
Related concepts
- Message queues
A durable buffer between producers and consumers that hands each message to one consumer at a time and redelivers it unless acknowledged.
- Persistent connections
Long-lived connections such as WebSockets turn a stateless request tier into one that holds per-client state, with consequences for routing, deploys and failure detection.
- Server push: polling, long polling, SSE, WebSockets
The ways a server can tell a client that something changed, and how update rate, latency needs and who-knows-what decide between them.
- Ordering
There is no global 'now' in a distributed system. Order exists only where something assigns it, so decide which order you need and who assigns it.
- Fan-out on write and fan-out on read
When one write must reach many readers, do the work when it is written (precompute every reader's view) or when it is read (assemble it on demand). Most real feeds do both.