Bidirectional Streaming and Flow Control
Understand how to manage continuous data streams in both directions and implement basic flow control.
Bidirectional Streaming and Flow Control is a free WebSockets & Realtime Systems Programming lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the WebSockets & Realtime Systems Programming learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Beyond Simple Messages
So far, we've mostly thought about WebSockets as a way to send discrete messages back and forth. But what if you need to transfer a continuous stream of data?
This is where bidirectional streaming comes in. It's about maintaining a steady, ongoing flow of data simultaneously in both directions, not just isolated messages.
Why Bidirectional Streaming Matters
Imagine scenarios like:
- Live Audio/Video: Sending and receiving real-time media streams.
- Large File Transfers: Uploading or downloading big files in chunks.
- Real-time Analytics: Continuous updates for dashboards with high data volume.
For these, a constant 'river' of data is more efficient than many small, separate 'droplets'.
The Challenge of Data Flow
What happens if one side sends data much faster than the other can process it? Think of a firehose pouring water into a small cup.
The receiver's temporary storage (called a buffer) will quickly fill up. This can lead to:
- Data loss
- System slowdowns
- Memory exhaustion
This is a critical problem for continuous data flow.
Introducing Flow Control
To prevent overwhelming a receiver, we use Flow Control. It's a mechanism that manages the rate of data transmission between a sender and a receiver.
Its main goal is to ensure the sender doesn't send data faster than the receiver can handle, making the communication smooth and reliable.
Key Flow Control Concepts
Flow control relies on a few core ideas:
- Buffering: Temporary storage for data that's been sent but not yet processed by the receiver.
- Backpressure: A signal from the receiver to the sender, indicating it needs to slow down or pause.
- Pause/Resume: Explicit commands or implicit behaviors to halt and restart data flow.
Think of it like a traffic light for your data stream.
WebSocket Buffering in Node.js
When you use ws.send(data) in Node.js, the ws library manages an internal buffer for outgoing messages.
The ws.send() method returns a boolean:
true: Data was sent immediately or buffered successfully.false: The internal buffer is full. You should pause sending new data.
You can also check ws.bufferedAmount, which tells you how many bytes are currently in the outgoing buffer.
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', ws => {
console.log('Client connected');
// Check buffer size before sending
const currentBuffer = ws.bufferedAmount;
console.log(`Current buffer size: ${currentBuffer} bytes`);
const ok = ws.send('Hello, streaming!');
if (!ok) {
console.log('Buffer full right away!');
} else {
console.log(`Message sent. New buffer size: ${ws.bufferedAmount} bytes`);
}
ws.on('message', message => {
console.log(`Received: ${message}`);
});
ws.on('close', () => console.log('Client disconnected'));
});
console.log('Server running on port 8080');Server-Side Backpressure (Part 1)
To implement flow control, your server needs to react when its outgoing buffer is full. If ws.send() returns false, you must stop sending data until the buffer clears.
This prevents the server from consuming too much memory or overloading the client's connection.
Here's the core idea of pausing:
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', ws => {
let messageCounter = 0;
let isPaused = false;
function sendLotsOfData() {
if (isPaused) return; // Don't send if paused
while (messageCounter < 1000) { // Send 1000 messages
const data = `Data chunk ${messageCounter++}`;
const ok = ws.send(data);
if (!ok) {
console.log('Buffer full! Pausing send...');
isPaused = true; // Set flag to pause
break; // Stop sending for now
}
}
if (messageCounter >= 1000) {
console.log('All data chunks sent!');
}
}
sendLotsOfData(); // Start sending
ws.on('message', msg => {}); // Placeholder
ws.on('close', () => {}); // Placeholder
});Server-Side Backpressure (Part 2)
When the buffer clears enough for more data, the ws library emits a 'drain' event. This is your cue to resume sending!
Combining ws.send()'s return value with the 'drain' event creates robust server-side flow control.
Try running this complete example. You'll see 'Buffer full!' and 'Buffer drained!' messages as flow control kicks in.
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', ws => {
console.log('Client connected. Starting data stream...');
let messageCounter = 0;
let isSending = false; // Flag to manage sending state
function streamData() {
if (isSending) return; // Already sending, wait for drain or completion
isSending = true;
while (messageCounter < 500) { // Simulate sending 500 messages
const data = `Stream chunk ${messageCounter++} from server.`;
const ok = ws.send(data);
if (!ok) {
console.log(`Buffer full (${ws.bufferedAmount} bytes), pausing send...`);
isSending = false; // Stop sending until drain
break; // Exit loop, wait for drain
}
}
if (messageCounter >= 500) {
console.log('Finished sending all stream chunks.');
isSending = false;
}
}
streamData(); // Start sending data after connection
ws.on('drain', () => {
console.log('Buffer drained, resuming send.');
streamData(); // Resume sending
});
ws.on('message', message => {
console.log(`Received from client: ${message}`);
// In a real app, client might also stream data here,
// requiring similar flow control logic on the client side.
});
ws.on('close', () => console.log('Client disconnected'));
ws.on('error', error => console.error('WebSocket error:', error));
});
console.log('WebSocket server started on port 8080. Connect a client to see streaming.');Client-Side Flow Control
While the server-side example focuses on outgoing data, clients also need to manage incoming streams and potentially their own outgoing streams.
On the client (browser JavaScript):
- For receiving data, you might buffer incoming messages if processing is slow.
- For sending large data (e.g., file uploads), you'd send in chunks and might need server-sent acknowledgments or explicit 'pause' signals to implement client-side backpressure.
The core principles remain the same: don't send faster than the receiver can handle.
Quick Check: Flow Control Logic
Consider a Node.js WebSocket server trying to send a large amount of data to a client. Which of the following statements about implementing server-side flow control are TRUE?
Recap: Mastering Realtime Streams
In this lesson, we explored bidirectional streaming and the crucial concept of flow control in WebSockets.
- Bidirectional streaming allows for continuous, simultaneous data flow, ideal for applications like live video or large data transfers.
- Flow control, using mechanisms like buffering, backpressure, and the
'drain'event, prevents overwhelming either the sender or receiver. - Implementing proper flow control is essential for building robust, high-performance, and reliable real-time applications.
Keep these principles in mind as you build your next streaming WebSocket application!
Frequently asked questions
Is the “Bidirectional Streaming and Flow Control” lesson free?
Yes — the full text of “Bidirectional Streaming and Flow Control” is free to read here on the web, and the WebSockets & Realtime Systems Programming course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the WebSockets & Realtime Systems Programming course, upgrade to CoddyKit PRO.
What will I learn in “Bidirectional Streaming and Flow Control”?
Understand how to manage continuous data streams in both directions and implement basic flow control. You practise WebSockets & Realtime Systems Programming with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start WebSockets & Realtime Systems Programming?
No prior experience is required. WebSockets & Realtime Systems Programming on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Bidirectional Streaming and Flow Control” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this WebSockets & Realtime Systems Programming lesson?
Yes. Every WebSockets & Realtime Systems Programming lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Implementing Publish/Subscribe Messaging
- Request-Response over WebSockets
- Bidirectional Streaming and Flow Control
- Backpressure and Message Batching