Cloudflare: rate limiting at the edge
Rate limits enforced in hundreds of data centres at once, where exact global counts would cost more than the attacks they stop.
The idea
A rate limiter that counts every request exactly, worldwide, needs coordination on every request. Cloudflare's design gives that up on purpose. Each data centre counts on its own. The sliding window is estimated from two fixed-window counters, weighted by how far into the current window you are. And once a client is over the limit, the decision to block is cached locally, so the counter is not touched again until it expires.
The part worth copying is that they measured the cost of approximating instead of assuming it, and found the error tiny. That is the argument to make in an interview: say what is approximate, how large the error is, and who it hurts.
Read the originals
Written by the engineers who built it.
- How we built rate limiting capable of scaling to millions of domains
Cloudflare · Post, Jun 2017
Approximating a sliding window with two counters, counting per data centre instead of globally, and measuring how wrong the approximation actually is.
Practise it
Make the decisions yourself, then compare.