API Rate Limiting with Token Bucket and Sliding Window

By Khimananda Oli | Last reviewed: August 2026

Unprotected APIs are a primary vector for denial-of-service attacks and runaway cloud bills in modern distributed systems. Effective API Rate Limiting with Token Bucket and Sliding Window algorithms provides the necessary guardrails to balance legitimate user access against abusive traffic patterns without degrading performance. This guide moves beyond theory to provide production-grade implementation patterns using Redis, helping you choose the right throttling strategy for your specific compliance and latency requirements.

How does the Token Bucket algorithm handle bursty API traffic?

The Token Bucket algorithm is the industry standard for API Rate Limiting with Token Bucket and Sliding Window because it gracefully handles real-world traffic patterns that are rarely perfectly uniform. Unlike rigid counters, it permits short bursts of activity up to a defined capacity while enforcing a long-term average rate. This behavior mirrors how actual users interact with applications: loading a dashboard might trigger twenty parallel API calls instantly, followed by minutes of silence.

Core mechanics and configuration parameters

A token bucket consists of three configurable variables that determine its behavior in production:

• Capacity (C): The maximum number of tokens the bucket can hold. This defines your burst limit. If C=100, a client can make 100 requests instantly after a period of inactivity.
• Refill Rate (R): Tokens added per second. This sets your sustainable throughput. R=10 means 10 requests/second on average.
• Current Tokens (T): The live count, always constrained between 0 and C.

When a request arrives, the system calculates tokens generated since the last check: T = min(C, T + (now - last_refill) * R). If T ≥ 1, the request proceeds and T decrements. Otherwise, return HTTP 429. For deeper context on protecting downstream databases from the resulting traffic patterns, see our MySQL performance tuning guide.

Production-ready Redis implementation

In distributed environments, you cannot store bucket state in application memory. Use Redis with a Lua script to guarantee atomicity. Race conditions between reading and writing token counts will silently corrupt your limits under load.

-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill_rate, ARGV[3] = now_ms
local key = KEYS[1]
local cap = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])

local data = redis.call('HMGET', key, 'tokens', 'last')
local tokens = tonumber(data[1])
local last = tonumber(data[2])

if tokens == nil then
  tokens = cap
  last = now
else
  local delta = math.max(0, now - last)
  tokens = math.min(cap, tokens + (delta * rate / 1000))
end

if tokens >= 1 then
  tokens = tokens - 1
  redis.call('HMSET', key, 'tokens', tokens, 'last', now)
  redis.call('EXPIRE', key, math.ceil(cap / rate) + 60)
  return {1, math.floor(tokens)}
else
  redis.call('HMSET', key, 'tokens', tokens, 'last', now)
  redis.call('EXPIRE', key, math.ceil(cap / rate) + 60)
  return {0, math.floor(tokens)}
end

This script executes atomically inside Redis. The TTL prevents orphaned keys from accumulating indefinitely. Always set expiry slightly longer than the time required to fully refill the bucket.

When should you use Sliding Window Log over fixed windows?

Fixed window counters suffer from the boundary burst problem: a client allowed 100 requests/minute could send 100 at :59 and another 100 at :00, creating a 200-request spike in two seconds. The Sliding Window Log eliminates this edge case entirely by tracking individual request timestamps within a rolling period. For compliance-sensitive workloads like payment processing or authentication endpoints where strict adherence to rate policies matters more than burst tolerance, this precision justifies the additional storage cost.

Atomic Sliding Window implementation with Redis Sorted Sets

Use Redis sorted sets (ZSET) where both member and score are the timestamp in milliseconds. This enables efficient range queries and automatic deduplication handling.

-- KEYS[1] = window key, ARGV[1] = limit, ARGV[2] = window_ms, ARGV[3] = now_ms
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cutoff = now - window

redis.call('ZREMRANGEBYSCORE', key, '-inf', cutoff)
local current = redis.call('ZCARD', key)

if current < limit then
  redis.call('ZADD', key, now, now .. '-' .. math.random(100000))
  redis.call('PEXPIRE', key, window)
  return {1, limit - current - 1}
else
  return {0, 0}
end

Note the random suffix appended to the timestamp member. Without it, multiple requests within the same millisecond would overwrite each other in the sorted set, causing undercounting. The PEXPIRE ensures keys self-clean even if cleanup logic fails. Teams managing observability for these patterns should correlate rate-limit metrics with their four golden signals monitoring dashboards.

What are the trade-offs between Token Bucket and Sliding Window?

Choosing between these algorithms requires understanding their operational characteristics beyond theoretical correctness. The following comparison reflects production realities observed across dozens of deployments:

In practice, many platforms implement both: Token Bucket for general API endpoints where developer experience matters, and Sliding Window for security-critical paths where predictability trumps flexibility. This hybrid approach appears frequently in architectures documented in our API gateways for microservices guide.

How do you implement distributed rate limiting without race conditions?

Single-instance rate limiting fails immediately in horizontal scaling scenarios. Distributed rate limiting centralizes state in Redis or Memcached, but introduces network latency and failure modes that must be addressed explicitly.

Critical implementation requirements

1. Atomic operations are non-negotiable. Never read-modify-write across separate Redis commands. Use Lua scripts or Redis 7.4+ functions to encapsulate the entire check-and-update in a single round trip.
2. Set defensive TTLs on every key. Orphaned rate limit keys consume memory indefinitely. TTL should exceed your window or refill period by a safety margin (typically 2x).
3. Implement graceful degradation. If Redis is unreachable, decide upfront: fail open (allow all traffic, risk abuse) or fail closed (block all traffic, risk outage). For most user-facing APIs, failing open with local in-memory fallback is preferable. For payment endpoints, fail closed.
4. Include rate limit headers in every response. Return X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. Clients need this information to implement backoff intelligently rather than hammering your API blindly.
5. Namespace keys rigorously. Include API version, endpoint path, and client identifier in the key schema. Example: ratelimit:v2:users:list:client_abc123. This prevents accidental collisions during deployments and enables granular policy changes.

Handling clock skew and timezone considerations

Distributed systems rarely have perfectly synchronized clocks. For Token Bucket, minor skew affects refill accuracy negligibly. For Sliding Window, skew can cause premature expiration or extended windows. Mitigate this by using Redis server time (TIME command) within Lua scripts rather than trusting client-supplied timestamps. In Nepal-based deployments serving global users, remember that BS calendar conversions are irrelevant at the infrastructure layer—always operate in UTC epoch milliseconds internally and convert only at presentation boundaries.

How do you monitor and tune rate limiters in production?

Deploying rate limits without observability is operating blind. You need visibility into three dimensions: effectiveness (are limits catching abuse?), user impact (are legitimate users being throttled?), and system health (is the rate limiter itself becoming a bottleneck?).

Instrument these metrics at minimum:

• Rejection rate by endpoint and client tier: Sudden spikes indicate either an attack or misconfigured limits. Track separately for free vs. paid tiers.
• P99 latency of rate limit checks: If your Lua script takes 5ms but your SLA budget is 50ms total, rate limiting consumes 10% of your error budget. Optimize aggressively.
• Token bucket depletion frequency: High depletion rates suggest capacity is too low for legitimate usage patterns. Adjust before users complain.
• Redis connection pool saturation: Rate limiters generate high-frequency, low-payload traffic. Monitor connection churn separately from application data connections.

Establish feedback loops with your support team. When legitimate users report 429 errors, correlate with your metrics to distinguish between genuine bugs and expected enforcement. Document tuning decisions in runbooks linked to your SLO-driven alerting configuration so future engineers understand why limits were set at specific values.

Securing Your API Surface with Intelligent Throttling

Implementing API Rate Limiting with Token Bucket and Sliding Window correctly requires matching algorithm choice to business intent, enforcing atomicity in distributed state stores, and maintaining continuous observability. Start with Token Bucket for general-purpose endpoints to preserve user experience, reserve Sliding Window for compliance-scoped paths, and never deploy either without comprehensive monitoring. If your team needs assistance designing rate limiting strategies that align with SOC 2 requirements or multi-region architectures, reach out to discuss your specific infrastructure challenges.