Rate Limiting

The question

Your API promises 100 requests a second per key, and ten gateway nodes enforce it. Depending only on how the load balancer spreads one client’s connections, that promise admits anywhere from 10 to 1,000 requests a second. Move the counter into a shared Redis and the number becomes exact, until the client speeds up and a 1 ms round trip quietly lets through twice the limit. Then Redis stops answering, and every node must decide, inside the request, whether to let everything through or nothing. Which of these numbers does your limiter actually produce?

The algorithms (token bucket, sliding windows) are one section of this page. The rest is about the part that breaks in production: many machines, one number, a network in between, and clients reacting to every 429 you send.

Why this problem exists

A rate limit is a promise about a key (an API key, a user, an IP address, or “this user on this endpoint”) over time: at most N requests in a window of W seconds, or an average rate with an allowed burst on top. The promise is made once, in your docs or pricing page. It is enforced by every process that receives traffic, each seeing only its own slice, and each deciding in well under a millisecond, because the limiter runs before every request.

The status code for “you, specifically, sent too many” is 429 Too Many Requests, defined in RFC 6585 (which also says a 429 must not be cached). The Retry-After header tells the client when to come back, as seconds or a date (RFC 9110).

Three things go wrong without a limiter, and each sets a requirement. A retry loop, a polling bug on a million phones or a scraper takes a share of capacity that belongs to everyone else, so the limiter must be fair per key. If requests call something paid (SMS, payment checks, a language model), excess requests become an invoice, so it must count what costs money, not only requests. And the limiter is now on the path of every request, so its own failure must not become the outage it was meant to prevent.

Rate limiting is often confused with its neighbours. They all reject or slow requests, for different reasons:

Idea	Limits	Decided by	Typical answer
Rate limit	Requests (or cost) per key per unit of time	The key’s recent behaviour	429 with Retry-After
Quota	Total usage per day or month, tied to a plan	The account	429 or 403 until the period resets
Concurrency limit	Requests per key in flight right now	What is running at this moment	429, or wait
Load shedding	Total work the server accepts	The server’s own health	503 for low-priority work

The line that matters: a rate limiter is about fairness between callers; load shedding is about the server’s survival. A million well-behaved clients each under their limit still take you down. You need both, and load shedding has its own page (coming soon).

The obvious fix, and why it breaks

The first limiter most of us write is a dictionary of counters in the app’s memory, keyed by the caller’s address and the current window number:

Python

naive_limiter.py

# runnable version: examples/rate-limiting/python/naive_limiter.py
LIMIT, WINDOW = 10, 10  # 10 requests per 10 seconds


class NaiveLimiter:
    """A dict of counters in this process's memory, keyed by (client, window number)."""

    def __init__(self) -> None:
        self.counts: dict[tuple[str, int], int] = {}  # never cleaned up: it grows forever

    def allow(self, client: str, now: float) -> bool:
        key = (client, int(now // WINDOW))
        self.counts[key] = self.counts.get(key, 0) + 1
        return self.counts[key] <= LIMIT

JavaScript

naive-limiter.mjs

// runnable version: examples/rate-limiting/js/naive-limiter.mjs
const LIMIT = 10;
const WINDOW = 10; // 10 requests per 10 seconds

/** A Map of counters in this process's memory, keyed by client and window number. */
export class NaiveLimiter {
  counts = new Map(); // never cleaned up: it grows forever

  allow(client, now) {
    const key = `${client}:${Math.floor(now / WINDOW)}`;
    const count = (this.counts.get(key) ?? 0) + 1;
    this.counts.set(key, count);
    return count <= LIMIT;
  }
}

The full example puts it through four situations every production system meets:

naive_limiter.py output

1. window boundary: 20 of 20 accepted within 0.1 s (limit is 10 per 10 s)
2. four servers:    40 of 40 accepted in one window (each server counted alone)
3. behind a proxy:  accepted asha 5/5, ben 5/5, chen 0/5 - three users share one counter
4. shared store:    20 of 20 accepted, and the stored count says 1

Each line is a different failure, and each gets a section below. Line 1 is the algorithm: ten requests at 9.9 s and ten at 10.0 s land in different windows. Line 2 is where the state lives: four copies of the app, four dictionaries, four times the limit. Line 3 is the key: behind a load balancer, the connection’s address is the balancer’s, so every user shares one counter. Line 4 is atomicity: twenty requests read 0 from a shared store before any of them writes, all twenty pass, and the store ends up saying 1. None of these shows up on a laptop.

The core idea

A rate limiter maps each request to a key, keeps a small piece of state per key, and decides before any expensive work is done. The algorithm chooses what that state is: two numbers for a token bucket, a counter per window, a list of timestamps. The architecture chooses where the state lives and who can see it. Almost every production failure of a limiter is one of four things: the wrong key, the state split across machines, the state seen late (a race or a sync delay), or the state unreachable (the store is down).

Enlarge figure

Counting alone with a share of the limit only adds up when the load balancer spreads every client evenly. One shared, atomic counter holds the limit however traffic is spread, at the price of a round trip and a dependency.

The top half has no dependency and no latency, and is right only if traffic splits evenly. The bottom half is exact, and every request now waits on a network call to a store that can be slow, contended or down. Real systems sit somewhere between the two, and most of this page is about where.

How it works, piece by piece

What a limiter has to remember

“100 requests per second” sounds like one number, but a rate is a count divided by a span of time, and a limiter never sees a rate. It sees one request arriving at one instant, and it has a fraction of a millisecond to answer one question: counting this one, has this key been too busy “recently”? Every algorithm below is a different definition of “recently”, and the definition decides what the limiter must store per key. There are two families.

Count inside a window. Pick a span of time, the window, and count the key’s requests that fall inside it. A fixed window cuts time into slices that start at fixed instants (10:00:00 to 10:00:10, then 10:00:10 to 10:00:20, and so on); each slice has its own counter, which starts again at zero when the next slice begins. A sliding window always looks back exactly W seconds from now, so the span moves with every request. With a limit of 10 per 10 seconds and a request arriving at 10:00:12:

Window	Requests it counts	What must be stored
Fixed, slice 10:00:10–10:00:20	Only those since 10:00:10	One counter per slice
Sliding, last 10 s	Those since 10:00:02	Every timestamp, or an estimate of them

Spend from an allowance that refills. Instead of counting the past, give each key an allowance and let requests spend it. One unit of allowance is called a token: a permission to make one request. Tokens are not objects anywhere; a token is a number going down by one. The bucket is only the name for where unspent allowance is kept, and it has a maximum size. Two numbers describe it:

• Capacity C, the most tokens the bucket can hold. This is the biggest burst a key can send at once after it has been quiet for a while.
• Refill rate r, how many new tokens appear each second. This is the average rate the key can keep up forever.

A mobile data plan that adds 1 GB a day and lets you carry over at most 10 GB works the same way: the daily top-up is the refill rate, the 10 GB ceiling is the capacity, and a quiet week lets you binge on one day but never by more than 10 GB. The analogy stops at the clock: the plan tops up once a day, while a token bucket refills continuously, in fractions of a token.

Here is a bucket with C = 10 and r = 1 token a second, for a key that has been idle:

Time	What arrives	Tokens before	Outcome	Tokens after
0.0 s	a burst of 10	10 (full)	all 10 accepted	0
0.0 s	an 11th	0	rejected	0
3.0 s	3 requests	0 + 3 s × 1 = 3	all 3 accepted	0
3.5 s	1 request	0.5	rejected; a whole token is 0.5 s away	0.5
60 s	1 request	would be 57, capped at 10	accepted	9

A leaky bucket is the same picture turned over. Instead of a bucket of permissions that requests drain, it is a bucket of requests that arrival fills and that leaks out at a fixed rate; a request that arrives to a full bucket overflows and is rejected.

So a limiter stores, per key, either counters or timestamps (window algorithms) or a token count plus the time it was last updated (bucket algorithms). That small piece of state is what the rest of this page moves between machines, races on, and loses when a store goes down.

The state per key: five algorithms in one pass

All examples use one limit: 10 requests per 10 seconds; for a bucket, a capacity of 10 refilling at 1 per second.

Token bucket. The allowance above, stored as two numbers. No timer drips tokens in: when a request arrives, add elapsed × r, cap at C, then try to take one, which is the arithmetic in the table above. This lazy refill makes the state two numbers (tokens, last update) and a million idle keys cost only storage. The retry time for a rejected request falls out of the same arithmetic: (1 − tokens) ÷ r.

Enlarge figure

A token bucket of 10 tokens refilling at 5 per second. Each request takes a token; a request that finds the bucket empty is rejected, and is told when the next token arrives.

The bound worth remembering: over any T seconds a token bucket admits at most C + r × T. That holds a surprise. “Capacity 10, refill 1 per second” sounds like “10 per 10 seconds”, but a full bucket followed by 10 seconds of refill admits up to 20 in 10 seconds (the comparison script below measured 19). A token bucket bounds the average rate and the burst size, not the count in every window. If your docs promise “at most 100 in any minute”, a window algorithm fits the promise better.

GCRA (generic cell rate algorithm, from ATM networks) is the same limit stored as one timestamp instead of two numbers: the theoretical arrival time (TAT), the moment at which the key would have used up all its allowance if it kept the average rate. Each accepted request pushes the TAT forward by 1 ÷ rate; a request is allowed if, after the push, the TAT is no more than burst ÷ rate seconds ahead of now. With 1 request a second and a burst of 3: three requests at t = 0 push the TAT to 1, 2 and 3 s, each no more than 3 s ahead, so all pass; a fourth would push it to 4 s, which is 4 s ahead, so it is rejected. At t = 1 s the TAT of 3 s becomes 4 s, which is again 3 s ahead, so one more request passes. One number per key matters when you keep millions of keys, and redis-cell implements it as one Redis command.

Leaky bucket names two different mechanisms, which is where most confusion in this topic comes from. As a meter, it is a level that drains at a fixed rate and rejects when full: the token bucket upside down, accepting exactly the same requests. As a queue, requests wait in a fixed-size FIFO and leave at a fixed rate: nothing is rejected until the queue is full, but accepted requests are delayed. A queue of 5 leaking 1 per second turns a burst of 8 into waits of 0, 1, 2, 3 and 4 seconds and 3 rejections. So the real choice is reject or delay: delay is right for a worker sending email through a provider capped at 10 per second, wrong for a user tapping a button. nginx’s limit_req is a leaky bucket whose burst is the queue; nodelay turns it into a meter.

Fixed window counter. One counter per key per window number (rl:{key}:{floor(now ÷ W)}), one INCR per request, and an expiry so old windows delete themselves. It is the cheapest algorithm there is, and it has the famous boundary problem.

Enlarge figure

Ten requests right before a window boundary count against window 1, ten right after count against window 2. Each counter says 10 of 10, so all twenty are accepted within a fraction of a second.

The less famous problems hurt more. Every key’s window resets at the same instant, so every client that was limited comes back at once and your traffic graph grows a spike at every boundary (the client section shows how Retry-After makes this worse). And a client that burns an hourly limit in its first second is locked out for 59 minutes.

Sliding window log. Store the timestamp of every accepted request (a Redis sorted set: remove entries older than W, count, add) and allow while fewer than N are inside the last W seconds. It is exact over every span, and its memory is O(limit) per key: 1,000 per hour for a million active keys is up to a billion timestamps, and even at a lean 16 bytes each that is 16 GB for one limit. Use it for small limits on valuable actions: 5 password resets an hour, 10 payouts a minute.

Sliding window counter. Keep two counters, the previous window and the current one, and assume the previous window’s requests were spread evenly:

estimate = previous × (1 − fraction of the current window elapsed) + current

Enlarge figure

With a limit of 100 per minute, 80 requests last minute, 30 so far this minute, and 40% of the way in, the sliding window still covers 60% of the previous minute: 80 × 0.6 + 30 = 78.

How wrong is “evenly spread”? Cloudflare compared it with exact counting on 400 million requests from 270,000 sources in 2017: 0.003% of requests were wrongly allowed or limited (Cloudflare). Adversarial traffic does worse: our comparison script puts ten requests at 9.9 s followed by a steady stream, and the counter admits 19 in one 10-second span. The error is spread over the window, though; the most in any single second stayed at 10.

Here is the token bucket on its own, then all five on the same traffic. Press “Send a burst of 20” in the first; in the second, press “Hit the window boundary”.

SimulationToken bucket

PlayStepReset

What to watch for. Press "Send a burst of 20". As many requests pass at once as there are tokens in the bucket, the rest bounce, and then requests trickle through again at the refill rate. The stored tokens are what let a burst through.

SimulationAll five on the same traffic

PlayStepReset

What to watch for. Hit the window boundary. Only the fixed window row lets the second group through, and its "most in 10 s" climbs to twice the limit. Then send a burst of 20: the token bucket lets ten through at once, the leaky bucket lets them out one per second.

In the compare run, only the fixed window row lets the second group of ten through, which is the 2× boundary effect with your own eyes. The token bucket lets a burst through at once and then trickles; the leaky bucket queue lets the same requests out one per second.

Many nodes, one limit

Put that state on N gateway nodes and you have three designs.

Each node counts alone, with the full limit. This is what you get by default from nginx’s limit_req, Envoy’s local rate limit filter, or a per-process library: the state is per instance (shared across the workers of one nginx or one Envoy, not across machines). A client spread over k nodes gets up to k times the limit.

Each node counts alone, with limit ÷ N. Now the sum is right only if traffic splits evenly, and it rarely does. HTTP/2 and gRPC clients hold one long-lived connection, so a layer 4 balancer pins the whole client to one node (the proxy and load balancer page shows why): a client paying for 100 a second gets 10. Autoscaling changes N without anyone changing the division, usually during the spike the limit was for. And randomness alone costs you: in the sim below, a client sending exactly 100 a second, spread evenly over 10 nodes that each allow 10, has about 12% of its requests rejected, because each node sees a random 10-ish per second, not exactly 10.

One shared counter. Every node asks the same store. The number is exact, and it costs a round trip per request (usually well under a millisecond inside one zone), a dependency in the request path, and a hot key when one large customer sends a big share of all traffic to one Redis key.

The middle ground: count locally, sync periodically. Each node admits against its last view of the shared total plus what it admitted itself since, and every S milliseconds adds its local count to the shared counter and reads the total back. Kong’s Rate Limiting Advanced plugin exposes exactly this knob as sync_rate: 0 means synchronous, -1 means local only, and a positive value is the sync interval in seconds, minimum 20 ms (Kong docs, as of October 2026). Its docs note the shared counters are accurate only when the sync is synchronous.

The error of the middle ground has a shape you can reason about. Between syncs, each node is blind to what the others admitted, so after the shared total reaches the limit, up to about client rate × S more requests get through before every node learns about it. A client sending 300 a second with a 200 ms sync gets about 60 extra per window. Here are all four on the same traffic:

SimulationOne limit of 100 req/s, enforced by N nodes

PlayStepReset

What to watch for. With the defaults (10 nodes, the client spread over all 10, sending 300 req/s), the first lane admits about 300 a second, three times the limit, and the synced lane about 155. Set the client to reach 1 node: the 100 ÷ N lane drops to 10, a tenth of what the client paid for, while the shared lane stays at 100. Move the sync interval and watch the synced lane's excess grow with it. Then take the store down: the shared lane fails open and admits everything for three seconds.

With the defaults, the four lanes average about 298, 100, 100 and 155 admitted per second (seed 42). Slide the sync interval: at 20 ms the synced lane admits about 107, at 100 ms about 128, at 500 ms about 232, close to the “client rate × S” estimate. Set the client to reach 1 node and the 100 ÷ N lane falls to 10 while the shared lane stays at 100. The shared lane is the only one that holds the promise in every setting, and the only one with a network call on every request.

Two production designs sit on this spectrum on purpose. Envoy recommends running its local limit (a token bucket per Envoy process) in front of its global limit (a gRPC call to a rate limit service backed by Redis): the local bucket absorbs bursts cheaply and protects the global service, and the global one gives the fleet-wide answer (Envoy). Cloudflare does not share counters between data centres at all: anycast routing sends one client to one data centre, so each counts on its own, and once a client is over the threshold a “mitigate” flag is cached on every server so later requests check a local flag instead of a counter.

From the trenches

Decide how wrong your limit may be before choosing the design. “Never more than 100 a second, ever” needs a shared atomic counter and its latency. “About 100 a second, a few percent over during bursts” is a synced counter, and costs a fraction of the round trips. Most public APIs are honestly the second; AWS’s API Gateway docs call its throttling limits “targets rather than guaranteed request ceilings”.

Check-then-act: the race that grows with load

A shared counter is exact only if the check and the update happen as one step. The naive version does GET count, compares it with the limit in the app, then SET count + 1. Between the store answering the GET and the SET arriving, a full round trip passes, and every other request whose GET lands in that gap reads the same, already stale, count. Each SET then overwrites the others, so the increments are lost too: that is how line 4 above ended with 20 admitted and a stored count of 1.

How much does it let through? Work it through at 1,000 requests a second with a 1 ms round trip. A new request arrives every millisecond, so while one request’s GET-then-SET is in flight, about one more request arrives and reads the same count. Every time the counter moves by one, about two requests were let through: one that was counted and one that read the stale value. In general each write hides the rate × round trip requests that arrive during it, so the admitted count is about limit × (1 + rate × round trip). At 100 requests a second and a 1 ms round trip, that is 100 × 1.1, invisible. At 1,000 a second it is 2× the limit, and at 3,000 a second, 4×. The race is not a rare corner case; it is a multiplier that grows with exactly the load the limiter exists for.

SimulationRead, check, write: how much the race lets through

PlayStepReset

What to watch for. At 1,000 req/s and a 1 ms round trip, the two-step lane admits about 200 a second, twice the limit. Drop the rate to 100 req/s, which is about what a test from a laptop sends: the race vanishes and both lanes look correct. Then raise the round trip to 5 ms, as when the store is in another zone or busy: the two-step lane admits about 600. The atomic lane stays at 100 whatever you do.

The sim’s defaults (1,000 req/s, 1 ms) admit about 198 a second in the two-step lane against 100 in the atomic lane; at 5 ms the two-step lane admits about 604. This is why a limiter can pass every test in staging, where one laptop sends 50 requests a second, and fail on the first busy day.

The fixes, in order of preference:

• For a counter, INCR is already atomic, meaning no other command can run between its read and its write: Redis executes commands one at a time on a single thread, so “read the number, add one, store it” happens as one indivisible step. It returns the new value, and you reject if it is over the limit. No read-then-write gap exists. The only trap is setting the expiry: INCR then EXPIRE as two commands leaves a counter with no expiry if the process dies between them, and that key is over the limit forever. Use one script, or SET key 0 EX 60 NX before the INCR.
• For state with several fields (a token bucket’s tokens and timestamp), the read, the decision and the write must run inside the store. A Lua script does that: Redis runs it from start to finish with no other command in between, in one round trip. Redis’s other two tools fall short here. MULTI/EXEC queues several commands and runs them together, but the app cannot look at a value read inside the batch and decide what to write. WATCH is optimistic locking: it aborts the batch if a watched key changed since you read it, so you retry, and a limiter’s hot key changes constantly, so under load it mostly retries (Redis tutorial).

Here is the token bucket as a Lua script, shared by the Python and JavaScript examples:

token bucket script (inside redis_limiter.py and redis-limiter.mjs)

local key      = KEYS[1]
local capacity = tonumber(ARGV[1])
local rate     = tonumber(ARGV[2])          -- tokens added per second
local cost     = tonumber(ARGV[3])
local t   = redis.call('TIME')              -- Redis's clock, not the app server's
local now = tonumber(t[1]) + tonumber(t[2]) / 1000000

local state  = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity   -- a new key starts with a full bucket
local ts     = tonumber(state[2]) or now
tokens = math.min(capacity, tokens + math.max(0, now - ts) * rate)

local allowed, retry_after = 0, 0
if tokens + 1e-9 >= cost then
  tokens = math.min(capacity, math.max(0, tokens - cost))   -- a negative cost is a refund
  allowed = 1
else
  retry_after = (cost - tokens) / rate
end

redis.call('HSET', key, 'tokens', tokens, 'ts', now)
-- Forget idle clients once their bucket would be full again anyway.
redis.call('PEXPIRE', key, math.ceil((capacity / rate) * 1000) + 1000)
return { allowed, tostring(tokens), tostring(retry_after) }   -- Lua numbers become integers

Four details matter more than they look. It reads Redis’s clock with TIME: two app servers whose clocks differ by 300 ms would disagree about how many tokens had refilled. It sets an expiry on every key, so quiet clients disappear. It returns strings, because Redis converts Lua numbers to integers on the way out and 3.7 tokens would arrive as 3. And the 1e-9 is there because ten refills of 0.1 add up to 0.9999999999999999; without it the token bucket and GCRA in the examples disagreed, 19 against 20.

Two costs of Lua. A script blocks the whole Redis shard while it runs, so keep it to a handful of commands; a slow script stalls every client of that shard, not only the limiter. And in Redis Cluster, which spreads keys over many servers by hashing each key into one of 16,384 slots, all keys a script touches must hash to the same slot, or the script is refused. A hash tag fixes that: only the part of the key inside {...} is hashed, so rl:{tenant42}:minute and rl:{tenant42}:day always land on the same server. Run against a real Redis, with twenty threads racing for ten tokens:

redis_limiter.py output (first lines)

connected to Redis 7.0.15
race, GET then SET: 20 of 20 accepted (limit 10)
race, Lua script:   10 of 20 accepted (limit 10)

The GET-then-SET line can change from run to run, because it depends on thread timing; the Lua line says 10 every time. A number that wobbles next to one that does not is what a race looks like from the outside.

When the store is slow or down

The shared counter is now a dependency of every request. When it stops answering, the limiter has three honest choices, and it must make one inside the request:

• Fail open: allow the request. The API stays up and is unprotected. Stripe and Envoy both default to this; Envoy’s global limit only rejects on errors if you set failure_mode_deny.
• Fail closed: reject. Right for limits that protect money or security: a login endpoint, OTP SMS sends, payouts. Wrong for a product catalogue, where your limiter has now caused an outage.
• Fall back to a local share: each node enforces limit ÷ N from memory until the store returns. Kong’s AI gateway documents the related behaviour of continuing to count locally when Redis is lost, so the fleet can exceed the limit, but each node stays bounded.

The fleet sim above has a “Take the store down for 3 s” button. Press it: the shared lane, which fails open, jumps from 100 to whatever the client sends; the synced lane falls back to each node’s local count and admits up to the full limit per node. With the client at 1,000 req/s, both admitted between 940 and 1,013 a second during the outage.

The example implements all three, logging the transition once instead of once per request (a limiter that logs every failed call during an outage adds a logging outage to the Redis one):

Python

redis_limiter.py

# runnable version: examples/rate-limiting/python/redis_limiter.py
def try_acquire(self, key: str, cost: int = 1) -> Result:
    try:
        allowed, tokens, retry = self.script(keys=[f"rl:{key}"], args=[self.capacity, self.rate, cost])
        self.down = False
        return Result(bool(allowed), float(tokens), float(retry))
    except redis.RedisError as err:
        if not self.down:  # log the transition once, not once per request
            log.warning("rate limiter store unavailable, failing %s: %s", self.when_down, err)
            self.down = True
    if self.when_down == "closed":
        return Result(False, 0.0, 1.0)
    if self.when_down == "local":
        # This node's share of the limit. Assumes the load balancer spreads clients evenly.
        bucket = self.local.setdefault(key, TokenBucket(self.capacity // self.nodes, self.rate / self.nodes))
        d = bucket.try_acquire(cost)
        return Result(d.allowed, float(d.remaining), d.retry_after)
    return Result(True, float(self.capacity), 0.0)  # open: the API stays up, unprotected

JavaScript

redis-limiter.mjs

// runnable version: examples/rate-limiting/js/redis-limiter.mjs
async tryAcquire(key, cost = 1) {
  try {
    const [allowed, tokens, retry] = await this.#run(key, cost);
    this.down = false;
    return { allowed: allowed === 1, tokens: Number(tokens), retryAfter: Number(retry) };
  } catch (err) {
    if (!this.down) console.warn(`WARNING ratelimit: rate limiter store unavailable, failing ${this.whenDown}: ${err.message}`);
    this.down = true; // log the transition once, not once per request
  }
  if (this.whenDown === 'closed') return { allowed: false, tokens: 0, retryAfter: 1 };
  if (this.whenDown === 'local') {
    // This node's share of the limit. Assumes the load balancer spreads clients evenly.
    if (!this.local.has(key)) this.local.set(key, new TokenBucket(Math.floor(this.capacity / this.nodes), this.rate / this.nodes));
    const d = this.local.get(key).tryAcquire(cost);
    return { allowed: d.allowed, tokens: d.remaining, retryAfter: d.retryAfter };
  }
  return { allowed: true, tokens: this.capacity, retryAfter: 0 }; // open: the API stays up, unprotected
}

That try is only half of failing open. The other half is how long the call takes to fail, and the client libraries’ defaults are built for caches and queues, where waiting is better than an error. Here is the same limiter pointed at a Redis that is not running, first with the library’s default client and then with retries off:

redis_limiter.py output (Redis down)

Redis down, default client: allowed=True after 4.55 s
Redis down, retries off:    allowed=True after 0.00 s
Redis down, 12 quick requests from one key:
  when_down=open    12 allowed
  when_down=closed   0 allowed
  when_down=local    5 allowed

The default redis-py 8.1 client, as installed for this page, retries a failed connection 10 times with exponential backoff capped at 1 s; across runs one call took 3.0 to 4.6 s to “fail open”, despite 50 ms socket timeouts. The default node-redis client is worse in a different way: it reconnects forever and queues commands while it does, so the same call had no answer after 3 s. A limiter that fails open after 4 seconds has not failed open; it has failed slowly, holding a worker for every request while Redis is down. Both examples build the client with timeouts in tens of milliseconds, retries off (Retry(NoBackoff(), 0) in Python) and, in Node, disableOfflineQueue and no reconnect loop.

Test the failure, not only the limit

Stop Redis in staging under load and watch the API’s latency, not only its error rate. The outage you are looking for is the one where every request waits for the limiter’s client to give up.

Who is the key?

The algorithm matters less than the key. A perfect sliding log keyed on the wrong thing protects nothing, or limits everyone at once. Each layer of a request’s path knows a different key, and the right limit lives where its key is known:

Enlarge figure

A request can be rate limited at every layer it passes through, and each layer keys on what it knows: the edge sees IP addresses, the gateway sees API keys, the service sees users and routes.

Behind any proxy, the socket address the app sees is the proxy’s. The client’s address arrives in X-Forwarded-For, which each proxy appends to, and which the client can also write. So the header is a list in which only the entries your own proxies appended are true:

Enlarge figure

Each hop appends the address it saw. The leftmost entry is whatever the client typed. Read from the right, skip addresses that belong to your own proxies, and the first one left is the real client.

Trusting the leftmost entry lets an attacker choose a fresh key for every request. Trusting the header from a connection that did not come from your proxy does the same. Trusting nothing means every user shares the load balancer’s address:

Python

redis_limiter.py

# runnable version: examples/rate-limiting/python/redis_limiter.py
# YOUR load balancers and CDN, nothing else. Real CDNs publish address ranges; check ranges then.
TRUSTED_PROXIES = {"127.0.0.1", "10.0.0.5", "198.51.100.2"}


def client_key(peer: str, api_key: str | None, forwarded_for: str | None) -> str:
    """Who to count. An API key beats an IP. X-Forwarded-For is only believed when the
    connection came from our own proxy, and only the entries our proxies appended."""
    if api_key:
        return f"key:{api_key}"
    if peer not in TRUSTED_PROXIES or not forwarded_for:
        return f"ip:{peer}"  # a direct connection: the socket address is the only truth
    hops = [hop.strip() for hop in forwarded_for.split(",")]
    for hop in reversed(hops):  # right to left: the newest entries were added by our proxies
        if hop not in TRUSTED_PROXIES:
            return f"ip:{hop}"  # the first address that is not one of ours
    return f"ip:{hops[0]}"

JavaScript

redis-limiter.mjs

// runnable version: examples/rate-limiting/js/redis-limiter.mjs
// YOUR load balancers and CDN, nothing else. Real CDNs publish address ranges; check ranges then.
const TRUSTED_PROXIES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1', '10.0.0.5', '198.51.100.2']);

/**
 * Who to count. An API key beats an IP. X-Forwarded-For is only believed when the connection
 * came from our own proxy, and only the entries our proxies appended.
 */
export function clientKey(peer, apiKey, forwardedFor) {
  if (apiKey) return `key:${apiKey}`;
  if (!TRUSTED_PROXIES.has(peer) || !forwardedFor) return `ip:${peer}`; // direct: the socket is the truth
  const hops = forwardedFor.split(',').map((hop) => hop.trim());
  // Right to left: the newest entries were added by our proxies. Take the first that is not ours.
  return `ip:${hops.findLast((hop) => !TRUSTED_PROXIES.has(hop)) ?? hops[0]}`;
}

output

key via LB:         ip:203.0.113.7
key, direct caller: ip:203.0.113.9

A real outage: the limiter that saw one client

On 24 October 2025, PostHog added an IP-based rate limit to its feature-flag endpoint. The middleware did not read X-Forwarded-For by default, so it saw every request as coming from the load balancer, and for 72 minutes about 97% of flag requests worldwide got a 429. It had been tested directly against the service, not behind the production load balancer, and nothing alerted on the 429 rate: detection took 62 minutes and came from customers (PostHog postmortem).

Even a correctly read IP is a weak key. Mobile carriers have far more phones than public IPv4 addresses, so they share addresses with carrier-grade NAT: the carrier’s router rewrites thousands of phones’ traffic to come from one public address. A per-IP limit then punishes all of them for one bad phone. IPv6 has the opposite problem. A home or a cloud server is normally given a whole /64, a block of 2^64 addresses that differ only in their last 64 bits, so one attacker can use a new address for every request. Per-address IPv6 limits are therefore usually applied to the whole /64. An API key is better, with its own trap: one customer running 200 legitimate servers on one key looks exactly like one abuser. Key on what you bill, and use IP limits only at the edge, against floods, where nothing else is known yet.

Count cost, not requests

A 1 ms cache read and a 30-second report export are both “one request”. When their costs differ by four orders of magnitude, a request count is a poor limit. Two answers exist.

Weighted tokens. Charge each request what it costs: the Lua script’s cost argument. GitHub’s REST API does this in its secondary limits: 900 points a minute, where most reads cost 1 point and most writes cost 5, alongside a cap of 100 concurrent requests (GitHub docs, as of October 2026).

Concurrency limits for long requests: count what is in flight, not what started. Stripe runs one next to its rate limiter, 50 concurrent requests in live mode as of October 2026, because a few slow endpoints exhaust workers long before they exhaust a per-second budget (Stripe).

Cost limits have a problem request counts never had: you often do not know the cost until the request is done. A language model’s reply length is unknown when the request arrives. Azure OpenAI, for example, documents that its tokens-per-minute limit is charged at arrival with an estimate built from the prompt and max_tokens, not from the tokens later billed (Microsoft Learn, as of October 2026). So set max_tokens close to what you need, or your limit fills with tokens you never use.

The general pattern is reserve, then refund: debit the worst case up front, then return the unused part when the real cost is known. The Lua script accepts a negative cost as a refund, capped at the bucket’s capacity:

redis_limiter.py output (cost)

cost: reserve 4,000 of 10,000  -> 6,000 left
cost: a 7,000 call now         -> 429, retry in 6.0 s
cost: refund the unused 2,400  -> 8,400 left, the 7,000 call is allowed

A request that reserved 4,000 tokens (1,000 of prompt plus max_tokens of 3,000) and used 600 of its 3,000 blocks a 7,000-token call until it refunds. Without the refund, the next call waits six seconds for tokens nobody used. With a fleet of nodes, the refund is one more round trip, and a node that crashes between reserve and refund leaks the reservation until the bucket refills.

429, Retry-After and the client on the other side

The limiter’s answer is a message to a program, and the program’s reaction decides whether the limit calms traffic or multiplies it.

Say 429, not 503. A 503 sends clients and dashboards looking for an outage and makes many clients retry harder; a 429 means “you, specifically”. nginx answers 503 by default, so set limit_req_status 429. Stripe adds a Stripe-Rate-Limited-Reason header naming the limit that was hit, worth copying. For state, the IETF draft draft-ietf-httpapi-ratelimit-headers-11 defines RateLimit-Policy: "default";q=10;w=5 (quota and window) and RateLimit: "default";r=0;t=5 (remaining and seconds to reset); as of October 2026 it is still a draft, and before it every API invented X-RateLimit-* headers with different meanings (X-RateLimit-Reset is a Unix timestamp at GitHub and “seconds from now” elsewhere).

Retry-After synchronises clients. With a fixed window, every client limited during a window is told to come back at the same boundary, so they all return in the same millisecond, and most are limited again. The fix belongs on both sides: servers that use windows can add a little random spread to Retry-After, and clients add jitter: wait at least Retry-After, plus a random extra that grows with each attempt (AWS Architecture Blog).

Do rejected requests count? If a 429 consumes nothing, a client in a tight retry loop keeps getting a slice through and never notices. If rejected requests count, a hammering client stays locked out until it stops, which teaches it. Discord goes further: 10,000 invalid requests in 10 minutes earns a temporary IP ban (as of October 2026). Pick one deliberately.

The example client sends 30 requests to an API limited to a burst of 5 and 5 a second, three ways: retrying at once, honouring Retry-After with jitter, and pacing itself with a client-side token bucket a little under the limit.

Python

client.py

# runnable version: examples/rate-limiting/python/client.py
def get_with_retry(url: str, polite: bool, attempts: int = 6) -> tuple[bool, int]:
    """Returns (succeeded, number of requests sent)."""
    for attempt in range(1, attempts + 1):
        status, retry_after = get(url)
        if status != 429:
            return status == 200, attempt
        if polite:
            backoff = min(8.0, 0.25 * 2**attempt)  # grows each time: 0.5, 1, 2, 4, 8 s
            # Retry-After is the earliest the server wants us back; jitter spreads clients out
            # so they don't all return in the same millisecond.
            time.sleep((retry_after or 0) + jitter.uniform(0, backoff))
    return False, attempts


class Pacer:
    """A client-side token bucket that waits for a token instead of failing."""

    def __init__(self, rate: float, burst: int) -> None:
        self.bucket = TokenBucket(capacity=burst, rate=rate)

    def wait(self) -> None:
        while not (decision := self.bucket.try_acquire()).allowed:
            time.sleep(decision.retry_after)

JavaScript

client.mjs

// runnable version: examples/rate-limiting/js/client.mjs
async function getWithRetry(url, polite, attempts = 6) {
  for (let attempt = 1; attempt <= attempts; attempt++) {
    const { status, retryAfter } = await get(url);
    if (status !== 429) return { succeeded: status === 200, attempts: attempt };
    if (polite) {
      const backoff = Math.min(8, 0.25 * 2 ** attempt); // grows each time: 0.5, 1, 2, 4, 8 s
      // Retry-After is the earliest the server wants us back; jitter spreads clients out
      // so they don't all return in the same millisecond.
      await sleep(((retryAfter ?? 0) + Math.random() * backoff) * 1000);
    }
  }
  return { succeeded: false, attempts };
}

/** A client-side token bucket that waits for a token instead of failing. */
class Pacer {
  constructor(rate, burst) {
    this.bucket = new TokenBucket(burst, rate);
  }

  async wait() {
    for (let d = this.bucket.tryAcquire(); !d.allowed; d = this.bucket.tryAcquire()) {
      await sleep(d.retryAfter * 1000);
    }
  }
}

client.py output

1. retry at once              5/30 ok  150 rejected  155 requests sent   0.2 s
2. Retry-After + jitter      30/30 ok    5 rejected   35 requests sent   6.3 s
3. client-side bucket 4.5/s  30/30 ok    0 rejected   30 requests sent   5.6 s

The impatient client sent five times the requests and finished a sixth of its work: every 429 it received cost the server a request anyway. The paced client got everything through with no rejections and finished first, which is why Stripe’s docs recommend a client-side token bucket for heavy integrations. Google’s SRE book describes a client-side version for whole services: adaptive throttling, where a client tracks its requests and the server’s accepts over two minutes and starts dropping requests locally once requests exceed twice the accepts, so a rejected request often never reaches the server at all (Google SRE book).

Choosing between them

The algorithm, configured as 10 per 10 seconds, on three traffic patterns (the “most in any 10 s” column is the worst we measured):

Algorithm	State per key	Bursts	Most in any 10 s	Adds latency?	Best for
Token bucket	2 numbers	Up to the bucket size	19	No	Most APIs; weighted costs
GCRA	1 timestamp	Same as token bucket	19	No	Token bucket semantics, millions of keys
Leaky bucket (queue)	Queue or 1 timestamp	Smoothed into a stream	10	Yes, by design	Protecting fragile downstreams
Fixed window	1 counter per window	Up to 2× at boundaries	20	No	Daily quotas, login attempts
Sliding log	1 timestamp per request	None beyond the limit	10	No	Small limits on valuable actions
Sliding window counter	2 counters	Mostly smoothed	19, spread out	No	Published per-minute limits at scale

And where the state lives, which matters more:

Design	Accuracy	Cost per request	When the store is down	Pick it when
Local, full limit	Up to k× over	None	Not affected	Edge flood protection, per instance
Local, limit ÷ N	Under when traffic is uneven	None	Not affected	Evenly spread, short-lived connections
Shared, atomic	Exact	One round trip	Your choice: open, closed, local	The limit is a contract or protects money
Local, synced every S	Over by about rate × S	One write per S per node	Keeps counting locally	High volume, a few percent over is fine

In practice I would start with a token bucket per API key in Redis, behind one Lua script, because its two knobs map to the questions product people ask (“how fast on average, how much burst?”) and it takes weighted costs. Put a local bucket at full limit in front of it at every node, sized as a safety net rather than the real limit, so a Redis outage degrades to “per node” instead of “unlimited”. Move to a synced counter when the round trips themselves become the cost.

Build it yourself

Every file is in examples/rate-limiting, in Python and JavaScript, and every output on this page came from running them:

• naive_limiter reproduces the four failures from the start of the page.
• limiters has all six algorithms with an injectable clock, and compare runs them on the same traffic.
• redis_limiter is the production-shaped version: the Lua script, the GET-then-SET race next to it, the key from X-Forwarded-For, 429 with Retry-After and the draft RateLimit fields, the reserve-and-refund demo, and the three behaviours when Redis is down. It needs Redis and the redis client library.
• client is the three clients.

The part of redis_limiter this page has not shown is the HTTP layer, and its output is the contract a client sees:

redis_limiter.py output (HTTP)

12 quick requests:  [200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 429, 429]
the 429 said:       Retry-After: 1   RateLimit: "default";r=0;t=5
                    RateLimit-Policy: "default";q=10;w=5

How real systems do it

Stripe described four layers in 2017: a request rate limiter (a token bucket per user in Redis), a concurrent requests limiter, and two load shedders acting on fleet health, one reserving capacity for critical traffic. The rollout advice is the part to copy: launch each limiter in observation mode, fail open, and keep a switch to turn it off (Stripe blog). As of October 2026 its API allows 100 requests a second in live mode and 25 for most endpoints.

GitHub moved its API limiter from Memcached, where unrelated cache data could evict live counters, to sharded Redis with Lua scripts and one primary per shard for writes (GitHub blog, 2021). The post also describes a reset header that wobbled by a second between requests, because the script returned a relative TTL and the app added it to its own clock; storing the absolute reset time fixed it. The lesson generalises: compute times the client sees from one clock.

nginx limit_req is a leaky bucket per key in a shared memory zone, shared by the workers of one nginx, not across nginx instances, so a fleet of nginx boxes is the “local, full limit” design. A working edge config:

nginx.conf

# runnable version: examples/rate-limiting/nginx/nginx.conf (run it with test-nginx.sh)
http {
    # One bucket per client IP, refilling at 5 requests per second. 10 MB of shared memory
    # holds about 80,000 clients on a 64-bit machine; when it is full, the least recently
    # used client is forgotten.
    limit_req_zone $binary_remote_addr zone=api:10m rate=5r/s;

    # nginx answers 503 by default, which says "we are broken". 429 says "you are too fast".
    limit_req_status 429;

    server {
        location /api/ {
            # Let up to 10 requests above the rate through at once (nodelay) instead of
            # queueing them; the 11th excess request in a burst is rejected.
            limit_req zone=api burst=10 nodelay;
        }
    }
}

test-nginx.sh output (nginx 1.24.0)

burst of 20:
  11 200
  9 429
one second later, 5 more:
  5 200

A burst of 20 gets 11 through (one at the rate plus the burst of 10), and a second later 5 more have refilled. Two details catch people: limit_req_dry_run on logs what would be rejected without rejecting, which is how to switch on any new limit; and a location that answers with return is never rate limited, because return runs in an earlier phase than limit_req.

Envoy and Kong were covered above: local in front of global, and a sync interval as a dial between accuracy and round trips. AWS API Gateway layers token buckets per client, per method, per account and per Region, and is candid that the result is best effort (AWS docs).

Gotchas and war stories

Launch every limit in observation mode. A new limit is a guess about real traffic. Run it in dry-run (nginx limit_req_dry_run, Envoy’s shadow mode, a “would reject” log line) for a week, look at who would have been limited, then enforce. The PostHog outage is what skipping this step looks like.

Alert on the 429 rate you send, per key and in total. A sudden rise in total 429s is your bug, not your customers’. A steady 429 rate for one key is a client that ignores Retry-After.

Local shares drift with autoscaling. limit ÷ N written into config at N = 10 is a 2× looser limit at N = 20 and a 2× tighter one if a zone fails and N drops to 5. Compute N at runtime from the service registry, or do not split.

Hot keys. One customer’s key can draw a large share of all limiter calls onto one Redis shard. The AWS Builders’ Library article on fairness suggests enforcing the very highest-volume keys locally, where they cannot overload a shared tracker (AWS Builders’ Library).

The limiter’s cost is not free either

Rejecting a request still costs a TLS handshake, parsing, a key lookup and a round trip to the store. For cheap endpoints, a flood of rejected requests costs almost as much as serving them, which is why the edge, not the application, absorbs floods.

Trade-offs and when not to use it

Every limiter adds a step, often a network call, to every request, and a dependency with failure modes of its own. Every key is unfair to someone: per IP punishes shared networks, per API key punishes the customer with many servers. Every fleet design trades accuracy for round trips, and the exact one is also the one that can take your API down with it.

It does not solve the problems it is often expected to solve. It is not overload protection: ten thousand clients each under their limit still overwhelm you, which is load shedding’s job. It is not DDoS protection: an attack from a million addresses sends a few requests from each and must be absorbed at the network edge. And it does not choose the numbers: teams usually start from observed traffic (what does the 99th-percentile customer send?) and add headroom.

When not to use it: inside a pipeline where you control both ends, such as a worker pulling from your own queue, backpressure and bounded queues are better, because the consumer pulls at its own pace and nothing is rejected. For an internal service with one or two callers, a concurrency limit plus load shedding needs less machinery.

Interview corner

“Design a rate limiter” is common because it fits 45 minutes and touches algorithms, distributed state, failure handling and API design. Strong answers ask about the requirements first and spend their time on the distributed part.

“Design a rate limiter for our public API, running on 50 gateway nodes.” Clarify the key, the limit, how exact it must be and the latency budget. Then: token bucket or sliding window counter per key in Redis, one Lua script per decision, Redis’s clock, keys with expiry. Strong candidates name the accuracy spectrum (local, shared, synced) and say which one the requirements pick.

“Why not GET the counter, check it and SET it back?” The race: every request that arrives during one round trip reads the same count, so admissions grow as limit × (1 + rate × round trip). INCR fixes a counter; a Lua script fixes multi-field state.

“Redis is down. What happens?” Fail open, closed or to a local share, chosen per limit by what an unlimited minute would cost. Then the part most candidates miss: millisecond timeouts and no client retries, or “fail open” means seconds of latency.

“How much memory for 10 million keys?” A token bucket in a Redis hash with key overhead is perhaps 100 bytes per key, about 1 GB: one instance. A sliding log at 1,000 entries per key does not fit. The sliding window counter is two integers.

Check yourself

1. Ten gateway nodes each allow 10 req/s for a key whose limit is 100. A gRPC client sends 80 req/s over one HTTP/2 connection. How many get through, and why?

About 10 a second. A layer 4 balancer assigns the single long-lived connection to one node, and that node allows its share of 10. The client is under its limit and still loses 87% of its requests. A shared counter, or per-request balancing at layer 7, fixes it.

2. A limiter uses GET, check, SET against Redis with a 2 ms round trip, limit 100 per second. A client sends 2,000 req/s. Roughly how many are admitted per second?

About 100 × (1 + 2,000 × 0.002) = 500, five times the limit. Every request arriving during one write’s round trip reads the same count. With INCR or a Lua script it would be 100.

3. Nodes count locally and sync to Redis every 500 ms. A client sends 300 req/s spread over all nodes against a limit of 100. Why does it get well over 100, and what are your two levers?

After the shared total reaches 100, every node keeps admitting until its next sync shows it the others’ counts, which can be up to 500 ms of the client’s traffic. The sim measured about 232 a second. Shorten the interval (more writes to Redis) or lower each node’s local allowance between syncs.

4. Your limiter fails open and Redis goes down. Latency on the whole API jumps to 4 seconds. Error rates are fine. What happened?

The Redis client retried each failed call with backoff before raising, so every request waited seconds for the limiter to give up before failing open. Configure the limiter’s client with timeouts in tens of milliseconds and no retries, and test it by stopping Redis under load.

5. A language model API limits tokens per minute. Your requests use about 500 output tokens, but you send max_tokens of 8,000. What happens to your throughput, and why?

If the limit is charged at arrival from the prompt plus max_tokens, as Azure OpenAI documents, each request reserves about 16 times what it uses, so you hit the limit after a sixteenth of the real usage. Set max_tokens near what you need. On your own limiter, reserve and then refund.

Recap

• A limit is a promise about a key, kept by many machines that each see part of the traffic. The algorithm is the small part.
• Local counters admit up to k× the limit; limit ÷ N under-admits whenever traffic is uneven; a shared atomic counter is exact for a round trip; syncing every S overshoots by about rate × S.
• GET then SET admits about limit × (1 + rate × round trip). Use INCR for counters and one Lua script for bucket state, on Redis’s clock, with an expiry.
• Choose fail open, closed or local share per limit, and make the client fail in milliseconds: default client retries turn “fail open” into seconds of latency.
• Key on what you bill. Read X-Forwarded-For from the right, only from your own proxies.
• Count cost when requests differ, reserve then refund when the cost is unknown, answer 429 with Retry-After, and expect clients to synchronise unless someone adds jitter.

Where to go next

• Load sheddingcoming soon

  When the whole system is overloaded, not one client: rejecting work by priority to stay alive.

  Resilience
• Circuit breakerscoming soon

  The mirror image of a limiter: protecting yourself from a dependency that is failing, such as the limiter's own store.

  Resilience
• Backpressurecoming soon

  When you control both ends of a pipeline, slowing the producer beats rejecting its work.

  Messaging & async
• Design a rate limiter servicecoming soon

  The full case study: a rate limiter as a shared service for many teams, with rules, storage and failover.

  Case studies
• LLM rate limits and token budgetscoming soon

  Limits in tokens per minute, estimated at arrival: reserve and refund applied to AI APIs.

  AI-era systems
• REST API Design→

  The rest of the HTTP contract around a 429: retry semantics per method, idempotency keys, status codes a client acts on, and Problem Details errors.

  Networking & APIs
• Reverse Proxy vs Load Balancer vs API Gateway→

  Where the gateway sits, why long-lived connections pin clients to one node, and what X-Forwarded-For you can trust.

  Traffic & edge

Further reading

Specs & official docs

• RFC 6585 §4, 429 Too Many Requests — the status code, and the rule that a 429 must not be cached.
• RFC 9110 §10.2.3, Retry-After — the two formats the header can take.
• IETF draft: RateLimit header fields for HTTP — RateLimit and RateLimit-Policy (version 11, May 2026).
• nginx ngx_http_limit_req_module — burst, nodelay, dry_run and the 503 default.
• Envoy global rate limiting and local rate limit filter — local in front of global, and failure_mode_deny.
• Kong Rate Limiting Advanced — sync_rate, the accuracy-for-round-trips dial.
• Redis: build 5 rate limiters with Redis — each algorithm as a Lua script, and why Lua beats MULTI and WATCH.
• redis-py retry helpers — the client’s retry and backoff configuration.
• AWS API Gateway throttling — layered token buckets, best effort.
• GitHub REST API rate limits, Stripe rate limits and Discord rate limits — points, concurrency limits, and invalid-request bans in practice.
• Azure OpenAI quotas — tokens per minute charged from an estimate at arrival.

Engineering blogs

• Stripe: Scaling your API with rate limiters (2017) — four layers of limiting and shedding, and how to roll them out.
• Cloudflare: Counting things, a lot of different things (2017) — the sliding window counter, its measured error, and per-data-centre counting.
• GitHub: A sharded, replicated rate limiter in Redis (2021) — moving off Memcached, and the wobbling reset header.
• AWS Builders’ Library: Fairness in multi-tenant systems — three ways to share a limit across a fleet, and hot keys.
• AWS Architecture Blog: Exponential backoff and jitter — why synchronised retries need randomness.
• Google SRE book: Handling overload — client-side adaptive throttling and the cost of rejecting.
• PostHog: feature flags outage postmortem (2025) — the limiter keyed on the load balancer.

Explainers

• Brandur Leach: Rate limiting, cells, and GCRA — GCRA explained, and why fixed windows frustrate users.
• smudge.ai: Rate limiting algorithms — interactive demos of windows and buckets.