In-Memory Rate Limiter Design
Design and implement a basic in-memory rate limiter, suitable for single-instance applications, using common programming patterns.
In-Memory Rate Limiter Design is a free API Rate Limiting & Scalability Patterns lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the API Rate Limiting & Scalability Patterns learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Intro to In-Memory Limiting
Welcome to designing an in-memory rate limiter! This is the simplest type of rate limiter, perfect for understanding the core concepts.
An in-memory rate limiter stores all its tracking data (like how many requests a user has made) directly in the application's RAM, not in a separate database or service.
This makes it fast and easy to set up, but it comes with specific limitations we'll explore.
Why Use In-Memory?
In-memory rate limiters are ideal for:
- Single-instance applications: Where your application runs on just one server.
- Quick prototypes: To test rate limiting concepts without complex infrastructure.
- Non-critical APIs: Where occasional dropped requests due to server restarts are acceptable.
They are simple to implement because they don't need to communicate with external data stores.
Core Design Concepts
Every rate limiter needs to track a few key pieces of information:
- Client ID: Who is making the request? (e.g., IP address, user ID, API key)
- Request Limit: How many requests are allowed? (e.g., 100 requests)
- Time Window: Over what period? (e.g., per minute, per hour)
Our in-memory design will use these concepts to decide if a request is allowed or denied.
Choosing a Strategy: Fixed Window
For our basic in-memory limiter, we'll use the Fixed Window Counter algorithm. It's straightforward:
- Requests are counted within a specific, fixed time window (e.g., 0-59 seconds, 60-119 seconds).
- When a new window starts, the counter resets to zero.
- If the request count exceeds the limit within the current window, new requests are denied.
While simple, it's a great starting point for understanding rate limiting mechanics.
Data Structures for Tracking
To keep track of requests for different clients within their time windows, we'll use Java's ConcurrentHashMap:
counts: A map to store the number of requests for eachclientId(e.g.,"user1" -> 5).windowStarts: A map to store the start time of the current window for eachclientId(e.g.,"user1" -> 1678886400000L).
ConcurrentHashMap is thread-safe, which is important when multiple requests might hit our limiter at the same time.
Building the Limiter Class
Let's start by defining our InMemoryRateLimiter class. It will hold our configuration (limit and window duration) and the maps for tracking.
Here's the basic structure:
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class InMemoryRateLimiter {
private final int limit; // Max requests allowed
private final long windowMillis; // Time window in milliseconds
private final ConcurrentHashMap<String, AtomicInteger> counts = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Long> windowStarts = new ConcurrentHashMap<>();
public InMemoryRateLimiter(int limit, long windowMillis) {
this.limit = limit;
this.windowMillis = windowMillis;
}
// The allowRequest method will go here
}Implementing `allowRequest` - Part 1
The heart of our limiter is the allowRequest(String clientId) method. This method will determine if a request from a given client should be allowed.
First, we get the current time and initialize the window start time for the client if it's their first request:
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class InMemoryRateLimiter {
private final int limit;
private final long windowMillis;
private final ConcurrentHashMap<String, AtomicInteger> counts = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Long> windowStarts = new ConcurrentHashMap<>();
public InMemoryRateLimiter(int limit, long windowMillis) {
this.limit = limit;
this.windowMillis = windowMillis;
}
public boolean allowRequest(String clientId) {
long currentTime = System.currentTimeMillis();
// Get or initialize window start time for this client
long currentWindowStart = windowStarts.computeIfAbsent(clientId, k -> currentTime);
// ... more logic to come ...
return false; // Placeholder
}
}Implementing `allowRequest` - Part 2
Next, we add the logic to check if the current time window has expired. If it has, we reset the window start time and the request count for that client.
This ensures that when a new window begins, clients get a fresh quota of requests.
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class InMemoryRateLimiter {
private final int limit;
private final long windowMillis;
private final ConcurrentHashMap<String, AtomicInteger> counts = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Long> windowStarts = new ConcurrentHashMap<>();
public InMemoryRateLimiter(int limit, long windowMillis) {
this.limit = limit;
this.windowMillis = windowMillis;
}
public boolean allowRequest(String clientId) {
long currentTime = System.currentTimeMillis();
long currentWindowStart = windowStarts.computeIfAbsent(clientId, k -> currentTime);
// If the current window has expired, reset it
if (currentTime - currentWindowStart >= windowMillis) {
windowStarts.put(clientId, currentTime); // Start a new window
counts.put(clientId, new AtomicInteger(0)); // Reset count
}
// ... more logic to come ...
return false; // Placeholder
}
}Implementing `allowRequest` - Part 3
Finally, we increment the request count for the client and check if it's still within the allowed limit. If it is, the request is allowed; otherwise, it's denied.
The AtomicInteger ensures thread-safe increments.
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class InMemoryRateLimiter {
private final int limit;
private final long windowMillis;
private final ConcurrentHashMap<String, AtomicInteger> counts = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Long> windowStarts = new ConcurrentHashMap<>();
public InMemoryRateLimiter(int limit, long windowMillis) {
this.limit = limit;
this.windowMillis = windowMillis;
}
public boolean allowRequest(String clientId) {
long currentTime = System.currentTimeMillis();
long currentWindowStart = windowStarts.computeIfAbsent(clientId, k -> currentTime);
if (currentTime - currentWindowStart >= windowMillis) {
windowStarts.put(clientId, currentTime);
counts.put(clientId, new AtomicInteger(0));
}
// Increment count and check if within limit
AtomicInteger clientCount = counts.computeIfAbsent(clientId, k -> new AtomicInteger(0));
if (clientCount.incrementAndGet() <= limit) {
return true; // Request allowed
} else {
return false; // Request denied
}
}
public static void main(String[] args) {
// Example usage will go here
}
}Full Example and Testing
Let's put it all together and test our in-memory rate limiter! This example creates a limiter allowing 3 requests per 5 seconds for a specific user.
Run the code and observe how requests are allowed initially, then denied, and finally allowed again after the time window resets.
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
public class InMemoryRateLimiter {
private final int limit;
private final long windowMillis;
private final ConcurrentHashMap<String, AtomicInteger> counts = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Long> windowStarts = new ConcurrentHashMap<>();
public InMemoryRateLimiter(int limit, long windowMillis) {
this.limit = limit;
this.windowMillis = windowMillis;
}
public boolean allowRequest(String clientId) {
long currentTime = System.currentTimeMillis();
long currentWindowStart = windowStarts.computeIfAbsent(clientId, k -> currentTime);
if (currentTime - currentWindowStart >= windowMillis) {
windowStarts.put(clientId, currentTime);
counts.put(clientId, new AtomicInteger(0));
}
AtomicInteger clientCount = counts.computeIfAbsent(clientId, k -> new AtomicInteger(0));
if (clientCount.incrementAndGet() <= limit) {
return true;
} else {
return false;
}
}
public static void main(String[] args) throws InterruptedException {
// Allow 3 requests per 5 seconds for "user1"
InMemoryRateLimiter limiter = new InMemoryRateLimiter(3, 5000);
String user = "user1";
System.out.println("Testing rate limiter for " + user + ": 3 requests / 5 seconds\n");
for (int i = 0; i < 5; i++) {
boolean allowed = limiter.allowRequest(user);
System.out.println("Request " + (i + 1) + ": " + (allowed ? "Allowed" : "Denied"));
if (i == 2) { // After 3rd request, wait for window to reset
System.out.println("\n--- Max requests reached. Waiting for window reset (5.5s) ---\n");
Thread.sleep(5500); // Wait for window to reset
}
}
System.out.println("\n--- Testing after window reset ---\n");
for (int i = 0; i < 2; i++) {
boolean allowed = limiter.allowRequest(user);
System.out.println("Request " + (i + 1) + ": " + (allowed ? "Allowed" : "Denied"));
}
}
}Understanding In-Memory Limitations
While simple and fast, in-memory rate limiters have a critical limitation. Imagine you deploy your application on multiple servers to handle more traffic.
What happens if requests from the same user go to different servers?
Recap: In-Memory Rate Limiting
You've successfully designed and understood a basic in-memory rate limiter!
- We defined an in-memory rate limiter and its use cases for single-instance apps.
- We explored key concepts: client ID, limit, and time window.
- We implemented a Fixed Window Counter using
ConcurrentHashMapin Java. - You now understand its primary limitation: it's not suitable for distributed systems due to its lack of shared state.
This foundational knowledge is crucial before diving into more advanced, distributed rate limiting solutions!
Frequently asked questions
Is the “In-Memory Rate Limiter Design” lesson free?
Yes — the full text of “In-Memory Rate Limiter Design” is free to read here on the web, and the API Rate Limiting & Scalability Patterns course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the API Rate Limiting & Scalability Patterns course, upgrade to CoddyKit PRO.
What will I learn in “In-Memory Rate Limiter Design”?
Design and implement a basic in-memory rate limiter, suitable for single-instance applications, using common programming patterns. You practise API Rate Limiting & Scalability Patterns with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start API Rate Limiting & Scalability Patterns?
No prior experience is required. API Rate Limiting & Scalability Patterns on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “In-Memory Rate Limiter Design” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this API Rate Limiting & Scalability Patterns lesson?
Yes. Every API Rate Limiting & Scalability Patterns lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- In-Memory Rate Limiter Design
- Distributed Rate Limiting with Redis
- Handling Rate Limit Exceedance
- Testing and Monitoring Your Rate Limiter