Rate Limiting and Quota Management
Per-user, per-org, and per-endpoint quotas so one tenant can't burn your OpenAI budget.
Rate Limiting and Quota Management is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Limits?
Without limits, a single abusive client can:
- Burn through your OpenAI budget
- Crowd out other users
- DDoS your service
Rate limits and quotas protect cost, latency, and fairness.
Rate Limit vs Quota
- Rate limit — requests per second/minute (short term)
- Quota — total budget per day/month (long term)
You need both.
Token Bucket Algorithm
The classic algorithm: each user has a "bucket" that refills at a fixed rate. Each request takes one token. No tokens, no service.
Redis Token Bucket Example
def allow(user_id, rate=10, capacity=30):
key = f'rate:{user_id}'
now = time.time()
pipe = redis.pipeline()
pipe.hgetall(key)
state, _ = pipe.execute()
tokens = float(state.get('tokens', capacity))
last = float(state.get('last', now))
tokens = min(capacity, tokens + (now - last) * rate)
if tokens < 1:
return False
tokens -= 1
redis.hset(key, mapping={'tokens': tokens, 'last': now})
redis.expire(key, 60)
return TrueFastAPI Integration with slowapi
# pip install slowapi
from slowapi import Limiter
from slowapi.util import get_remote_address
limiter = Limiter(key_func=get_remote_address)
app.state.limiter = limiter
@app.post('/agents/qa')
@limiter.limit('10/minute')
def qa(req: AgentRequest):
...Different Limits Per Tier
class User:
def __init__(self, is_pro):
self.is_pro = is_pro
class State:
def __init__(self, is_pro):
self.user = User(is_pro)
class Req:
def __init__(self, is_pro):
self.state = State(is_pro)
def limit_for(req):
return '100/minute' if req.state.user.is_pro else '10/minute'
def qa(req):
limit = limit_for(req)
print(f'user is_pro={req.state.user.is_pro} -> rate limit {limit}')
return limit
qa(Req(is_pro=True))
qa(Req(is_pro=False))
Cost-Based Quotas
Limit dollars, not just requests:
def check_budget(user_id, predicted_cost):
key = f'cost:{user_id}:{date.today()}'
spent = float(redis.get(key) or 0)
if spent + predicted_cost > daily_budget(user_id):
raise HTTPException(429, 'Daily budget exceeded')
# After the call:
redis.incrbyfloat(key, actual_cost)
redis.expireat(key, midnight_tomorrow_ts)Concurrency Caps
Limit in-flight requests per user too:
key = f'inflight:{user_id}'
redis.sadd(key, request_id)
if redis.scard(key) > 3:
raise HTTPException(429, 'Too many concurrent requests')
# Remove from set when the request finishes.Global Rate Limit
Protect downstream LLM APIs from your own service:
global_key = 'rate:global:llm'
if redis.incr(global_key) > 60: # 60 calls per second
raise HTTPException(503, 'Service overloaded')
redis.expire(global_key, 1)Return Useful Errors
headers = {
'X-RateLimit-Limit': '60',
'X-RateLimit-Remaining': '0',
'X-RateLimit-Reset': str(reset_time),
'Retry-After': '30'
}
return JSONResponse({'error': 'rate-limited'}, status_code=429, headers=headers)Distributed Rate Limits
If you run multiple API servers, rate limits must be shared. Redis is the standard backend; Cloudflare and Kong have managed alternatives.
Burst Allowances
Real usage is bursty. Allow a small burst (e.g. 30 requests at once if the rate is 10/sec). Token-bucket implementations handle this naturally with a higher `capacity`.
Alerting
Alert when:
- Global rate limit hits trigger more than X per minute
- Any user is consistently at their limit (likely needs upgrade or has a bug)
- Daily budget approaches 80%
Two-Layer Protection
Why have BOTH rate limits AND quotas?
Recap
Token bucket for rate limits, cost counters for quotas, concurrency caps per user, global circuit-breaker, helpful 429 responses with Retry-After.
Frequently asked questions
Is the “Rate Limiting and Quota Management” lesson free?
Yes — the full text of “Rate Limiting and Quota Management” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.
What will I learn in “Rate Limiting and Quota Management”?
Per-user, per-org, and per-endpoint quotas so one tenant can't burn your OpenAI budget. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents?
No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Rate Limiting and Quota Management” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents lesson?
Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Serving Agents Behind an API
- Async Workflows and Background Jobs
- Rate Limiting and Quota Management
- Blue-Green and Canary Deploys for Agents