Caching and Throttling

In the world of APIs, speed and reliability are key. High latency (slow responses) frustrates users, while sudden traffic spikes can crash your backend services.

This lesson explores two powerful features in AWS API Gateway: caching and throttling. They help you deliver fast, stable, and cost-effective APIs.

Caching is like having a temporary storage for frequently requested data. Instead of always fetching data from your backend (e.g., a database or Lambda function), API Gateway can store the response and serve it directly.

• Reduces latency: Responses are returned much faster.
• Lowers load: Your backend services receive fewer requests.
• Saves costs: Less compute/database usage.

When caching is enabled for an API stage, API Gateway stores responses from your backend for a specified duration.

Here's how it works:

1. A client sends a request to API Gateway.
2. API Gateway checks if a valid response for that request is in its cache.
3. If found, API Gateway immediately returns the cached response.
4. If not found, API Gateway forwards the request to your backend, caches the response, then returns it to the client.

Caching is configured at the API stage level. You specify a cache capacity and a default Time-to-Live (TTL) for cached responses.

• Cache Capacity: The size of the cache (e.g., 0.5 GB to 237 GB).
• TTL (Time-to-Live): How long responses are stored in the cache, in seconds (default 300s).
• Encryption: Cache data can be encrypted at rest.

You can also override TTL per method or even disable caching for specific methods.

What if your backend data changes? You don't want to serve stale cached data. This is where cache invalidation comes in.

• Client-side: Clients can send a Cache-Control: max-age=0 header to bypass the cache and force a fresh response.
• API Key: If using API keys, clients with the key can invalidate the cache for specific requests.
• Manual: You can manually invalidate the entire stage cache via the AWS Management Console or CLI.

Throttling is about limiting the number of requests your API can receive over a certain period. Think of it as a traffic cop for your API.

It's crucial for:

• Backend Protection: Prevents your backend services from being overwhelmed.
• Cost Control: Reduces unexpected spikes in resource usage.
• Fair Usage: Ensures all users get a reasonable share of API access.

API Gateway enforces throttling based on two main metrics:

• Rate: The steady-state number of requests per second (RPS) that clients can submit to your API.
• Burst: The maximum number of concurrent requests API Gateway will fulfill before returning 429 Too Many Requests errors. This allows for brief spikes above the steady rate.

When limits are hit, API Gateway rejects excess requests, protecting your backend.

AWS sets default account-level throttling limits for API Gateway to ensure service stability across all users. These are soft limits, meaning you can request increases.

• Example: A default might be 10,000 requests per second (RPS) and a burst of 5,000 concurrent requests across all APIs in a region.

These limits apply before any stage-specific or usage plan limits are considered.

Beyond account-level limits, you can configure stage-level throttling for individual methods or entire API stages.

• You can set a default rate limit and burst limit for the entire stage (e.g., 100 RPS for dev, 1000 RPS for prod).
• You can also override these defaults for specific HTTP methods (e.g., a POST /items might have a lower limit than a GET /items).

For more granular control, especially when managing different user tiers or monetizing your API, you use Usage Plans combined with API Keys.

• Usage Plan: Defines custom throttling limits (rate, burst) and quotas (total requests over a period) for a group of clients.
• API Key: A unique identifier that clients include in their requests. API Gateway uses this key to map the client to a usage plan and enforce its specific limits.

This allows you to offer different service levels (e.g., Free, Premium) to different clients.

Understanding the purpose of throttling is crucial for building resilient APIs.

We've explored how caching dramatically improves API performance by reducing latency and backend load. We also learned how throttling protects your services from traffic spikes and ensures fair usage through account-level, stage-level, and usage plan configurations.

These features are essential for building robust and scalable serverless APIs with AWS API Gateway.