14.1System Design Building Blocks

Rate Limiter

Control the rate of incoming requests using different algorithms. Compare Token Bucket, Leaky Bucket, Fixed Window, Sliding Window Log, and Sliding Window Counter approaches.

Algorithm
Presets
Limit:5
Refill:1/tick
Token Bucket
10
/ 5
Refill rate: 1 token/tick
Request Flow
Client
Limiter
Service
Timeline (last 60 ticks)
AcceptedRejected
Timeline events will appear here
0now (0)
Acceptance Rate0.0%
Accepted: 0Rejected: 0
Event Log
No events yet. Press Play or Step to begin.
1.0x
Total Requests0
Accepted0
Rejected0
Tokens10
Acceptance Rate0.0%
Tick0
Understanding Token Bucket

How It Works

Tokens are added to a bucket at a fixed rate. Each request consumes one token. Requests are rejected when the bucket is empty. Allows bursts up to bucket capacity.

Advantages

  • Allows controlled bursts
  • Simple to implement
  • Memory efficient

Limitations

  • Burst size limited by bucket capacity
  • No guarantee of smooth rate
Key InsightToken Bucket is used by AWS API Gateway, Stripe, and most cloud providers. It naturally handles bursty traffic by accumulating tokens during quiet periods. The burst capacity equals the bucket size, and the sustained rate equals the refill rate.

Algorithm Comparison

AlgorithmMemoryBurstAccuracyUsed By
Token Bucket
O(1)YesGoodAWS, Stripe
Leaky Bucket
O(1)NoGoodNGINX
Fixed Window
O(1)EdgeLowRedis
Sliding Log
O(n)NoExactFinance
Sliding Counter
O(1)NoHighCloudflare

Real-World Applications

Rate limiting is essential in API gateways (Kong, AWS API Gateway), CDNs (Cloudflare, Akamai), social media platforms (Twitter/X rate limits), payment processors (Stripe), and any public-facing service. It prevents abuse, ensures fair usage, protects against DDoS attacks, and helps maintain service reliability under load. Most implementations use a combination of algorithms at different layers: token bucket for per-user limits and sliding window for global rate limiting.