Rate Limiting

Rate limiting protects a service from being overwhelmed, whether by abuse, by a buggy client in a retry loop, or by one tenant crowding out the rest. The algorithm choice trades burst tolerance against memory.

Fixed window counts requests per clock interval. Trivial to implement, but it allows double the limit across a boundary: 100 requests at 11:59:59 and 100 more at 12:00:00 is 200 in one second.

Sliding window log stores a timestamp per request and counts those inside the window. Perfectly accurate, and memory grows with request volume. Sliding window counter interpolates between the previous and current fixed window, which is nearly as accurate for a fraction of the memory. That is why it is the common production choice.

Token bucket refills tokens at a steady rate up to a cap, and each request spends one. It permits bursts up to the bucket size while bounding the sustained rate, which usually matches what you actually want from an API. Leaky bucket drains at a fixed rate and smooths bursts away entirely, which is better when the thing downstream genuinely cannot absorb a spike.

Distributed limiting is where it gets hard. Per-instance counters let the real limit drift to limit x instances. A shared store (typically Redis) gives one global count at the cost of a network hop on every request. Return 429 with a Retry-After header so well-behaved clients back off instead of hammering.

Not every request costs the same. A limit counted in requests is easy to reason about and wrong for an API where one call reads a row and another runs a report. Cost-based limiting charges a weight per operation against the same bucket, which is what GraphQL query-cost limits and cloud provider quotas do. It takes more work to calibrate, and it is the only approach that stops your single most expensive endpoint from being the whole attack surface.

Decide in advance what happens when the limiter itself fails. If the shared counter is unreachable, failing closed rejects legitimate traffic because a component that exists purely to protect you is down; failing open removes the protection at exactly the moment load is unusual. Most services fail open and drop back to a coarse per-instance limit, which bounds the worst case without letting a Redis blip become an outage. Either is defensible; choosing by accident is not.