Rate-limiting at scale (at Cloudflare)

Doing this with possibly millions of domains and even more millions of rules immediately becomes a bit more complicated.

They ended up with a simple yet fairly accurate approximation of the number of requests per ip, based around a counter stored in memcached, incremented asynchronously. There’s quite a bit of other fun stuff, like how they ensure traffic goes to the same server (anycast?), and the other algorithms they went through.