Rate Limiting

A rate limiter caps how many requests a client can make in a time window, protecting your system from abuse, runaway bugs, and thundering herds. Requests over the limit get HTTP 429 ("too many requests") plus a Retry-After header telling the client when to try again.

The animation shows the token bucket — the industry default, used by AWS and Stripe. A bucket holds up to B token coins and refills at a steady drip of R tokens per second. Every request must grab a coin to pass. Watch three phases:

1. Normal traffic: requests arrive slower than the refill rate, always find a coin, and sail through.
2. A burst: a rapid volley drains the bucket. The first B requests all pass — this is the token bucket's selling point: short bursts are allowed, up to the bucket's capacity.
3. Over the limit: with the bucket empty, requests bounce off with red 429 badges until the drip restores coins and traffic resumes.

Two interview follow-ups to have ready. Where does it live? At the API gateway, checking a counter in Redis shared by all gateway instances. And why must the check-and-take be atomic (a single Lua script, not a read followed by a write)? Because two gateways reading "1 token left" at the same moment would both let their request through.