Throttling, Caching, and Usage Plans

Without throttling, a single misbehaving client or a traffic spike could overwhelm your backend services—Lambda concurrency, RDS connections, or downstream APIs. Throttling in API Gateway limits the number of requests per second and allows short bursts above the steady-state rate. Throttled requests receive a 429 Too Many Requests response immediately, without the request reaching your backend, protecting downstream resources from overload.

Throttling operates at multiple levels. The account-level limit is 10,000 requests per second (RPS) with a burst of 5,000 requests (soft limit, can be increased). At the stage level, you can set a default throttle rate (RPS) and burst limit that applies to all methods in the stage. At the method level, you can override the stage defaults for specific endpoints—for example, giving a read-heavy GET endpoint a higher rate limit than a write-heavy POST endpoint.

API Gateway throttling uses a token bucket algorithm. Tokens accumulate in a bucket up to the burst limit (maximum instant capacity). Each request consumes one token. Tokens refill at the rate limit (steady-state RPS). If the bucket is empty, requests are throttled. Example: burst=5000, rate=1000 RPS. At the start you can handle 5000 simultaneous requests; the bucket refills at 1000 tokens/second. This allows absorbing brief traffic spikes while enforcing long-term rate limits.

Response caching (available on REST API stages) stores backend responses in an API Gateway-managed cache so identical requests are served from cache without hitting the backend. This reduces backend load, lowers latency, and can significantly cut Lambda invocation costs for read-heavy APIs. Cache is keyed by the request (method, path, query strings, headers per configuration). The cache TTL is configurable from 0 to 3600 seconds (default 300 seconds).

By default the cache key is the full request URL. You can customise which elements contribute to the cache key: include specific query string parameters (e.g., pageSize, filter) but exclude irrelevant ones (e.g., timestamp). You can also include specific headers in the cache key. Exclude sensitive headers from the cache key to prevent private data from contaminating shared cache entries. Use cache key tuning to maximise cache hit rates while ensuring different logical requests get different cached responses.

Clients can invalidate the cache for a specific request by including the Cache-Control: max-age=0 header. You can also flush the entire stage cache from the console or API. Configure whether clients are allowed to invalidate cache—restrict this in production to prevent clients from bypassing caching intentionally. To grant flush permissions selectively, attach a resource policy or use a Lambda authoriser that checks if the caller has permission to flush.

A Usage Plan defines throttle limits and quota limits for a group of API clients. Associate an API stage with a usage plan and then associate API keys with the plan. Each API key enforces the plan's limits independently. Usage plans let you offer tiered access: a Free plan at 100 RPM/10,000 requests/day, a Pro plan at 1,000 RPM/100,000 requests/day. This is the model for monetised APIs and partner integrations where different clients need different rate limits.

API keys are opaque string tokens that clients include in the x-api-key request header. API Gateway validates the key and associates the request with the corresponding usage plan. API keys are NOT a security mechanism—they identify clients for throttling and quota purposes only. For security, always combine API keys with proper authorisation (IAM, Lambda authoriser, or Cognito). API keys that are not associated with a usage plan simply don't have throttle limits applied.

In addition to per-second throttle rates, usage plans support quota limits: a maximum number of requests over a time period (DAY, WEEK, or MONTH). Once a client exhausts their quota, subsequent requests return 429 until the quota resets. Quota limits are useful for free-tier enforcement, preventing API abuse, and aligning API consumption with billing. Quota counters are eventually consistent, so a client may exceed their quota slightly before being blocked.

Monitor API Gateway health with these CloudWatch metrics:

• Count: total API calls
• 4XXError: client errors including 429 throttles
• 5XXError: backend errors
• Latency: end-to-end request time
• IntegrationLatency: time waiting for backend
• CacheHitCount / CacheMissCount: cache effectiveness

Set alarms on 4XXError spikes to detect throttling issues before they impact users, and on CacheMissCount to identify cache configuration problems.

Use caching for read-heavy APIs where responses change infrequently—product catalogue lookups, reference data, static configurations. Caching is counterproductive for user-specific or highly dynamic data. Use throttling always—even for internal APIs—to protect backend services from overload. Combine both: cache common data to reduce backend load and throttle aggressively to prevent any single client from dominating the API. For the SAA-C03 exam, caching reduces cost and latency; throttling ensures availability.

Test your understanding of AWS Solutions Architect (SAA-C03) concepts from this lesson.

In this lesson you learned: Throttling at account, stage, and method levels protects backends using a token bucket algorithm with configurable rate and burst limits, Response Caching stores backend responses for configurable TTLs to reduce backend load and latency for read-heavy endpoints, and Usage Plans with API Keys enforce per-client throttle rates and quotas enabling tiered access control for partner and public APIs. Next up we explore ECS clusters, task definitions, and services for containerised workloads.