Pipeline Concepts: Stages, Operators, and Expressions
Learners will understand how data flows through pipeline stages and distinguish between stage operators and expression operators.
Pipeline Concepts: Stages, Operators, and Expressions is a free MongoDB Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MongoDB Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What Is the Aggregation Pipeline?
The aggregation pipeline is MongoDB's server-side data transformation engine. Instead of retrieving raw documents and processing them in application code, you define a sequence of stages that transform the data stream step by step—filtering, reshaping, grouping, computing, and sorting—all within the database engine. This keeps heavy computation close to the data and dramatically reduces what you send over the network.
// A simple aggregation pipeline
db.orders.aggregate([
{ $match: { status: 'completed' } }, // stage 1: filter
{ $group: { _id: '$userId', total: { $sum: '$amount' } } }, // stage 2: group
{ $sort: { total: -1 } }, // stage 3: sort
{ $limit: 10 } // stage 4: take top 10
]);How Documents Flow Through Stages
Think of the pipeline as a conveyor belt: each document enters stage 1, is transformed (or filtered out), and the results flow into stage 2, then stage 3, and so on. At each stage, the set of documents can shrink (filtering), expand (unwind), or be completely replaced by computed summaries (group). Documents that fail a filter condition are dropped from the pipeline and never reach later stages.
// Data flow visualization:
// Input: 10,000 orders
// Stage 1 ($match: status='completed'): 8,000 docs
// Stage 2 ($group by userId): 1,200 docs (one per user)
// Stage 3 ($sort by total DESC): 1,200 docs (reordered)
// Stage 4 ($limit 10): 10 docs
// Output to client: 10 docsStage Operators vs Expression Operators
The aggregation framework distinguishes between two kinds of operators: stage operators (prefixed with $) that define what a pipeline stage does—like $match, $group, $project—and expression operators (also prefixed with $) that compute values within a stage—like $sum, $avg, $concat. Stages are the building blocks; expressions are the calculations inside them.
db.sales.aggregate([
// $group is a STAGE operator
{ $group: {
_id: '$region',
// $sum and $avg are EXPRESSION operators (accumulators here)
totalRevenue: { $sum: '$amount' },
avgOrder: { $avg: '$amount' },
// $concat is an expression operator
label: { $concat: ['Region: ', '$region'] }
}}
]);Field References With the $ Sign
Inside aggregation expressions, a string prefixed with $ is a field reference—it refers to the value of that field in the current document. Without the prefix, a string is treated as a literal value. This distinction is crucial: '$price' means 'the value of the price field' while 'price' means the literal string 'price'.
db.products.aggregate([
{ $project: {
name: 1,
// '$price' references the 'price' field value
discountedPrice: { $multiply: ['$price', 0.9] },
// 'price' without $ is a literal string
label: 'price', // all docs get the string 'price', not the field value
// $$ROOT is a special variable for the whole document
original: '$$ROOT'
}}
]);System Variables: $$ROOT, $$NOW, $$CURRENT
The aggregation pipeline provides system variables prefixed with $$ that give access to special values: $$ROOT refers to the entire current document, $$NOW is the current datetime (useful in $project for computed fields), $$CURRENT is the current document field path context, and $$REMOVE conditionally removes a field when assigned as a value.
db.orders.aggregate([
{ $project: {
_id: 1,
orderDate: 1,
daysOld: {
$divide: [
{ $subtract: ['$$NOW', '$orderDate'] }, // $$NOW = current time
1000 * 60 * 60 * 24 // convert ms to days
]
},
// Conditionally remove a field:
sensitiveField: { $cond: { if: '$isAdmin', then: '$secret', else: '$$REMOVE' } }
}}
]);Common Pipeline Stages Overview
MongoDB provides dozens of pipeline stages. The most essential ones to know are: $match (filter documents), $project (reshape/compute fields), $group (aggregate by key), $sort (order results), $limit and $skip (pagination), $lookup (join another collection), and $unwind (flatten arrays). Master these and you can express almost any analytical query.
// Quick reference of the most-used stages:
// $match - { $match: { field: condition } }
// $project - { $project: { keep: 1, drop: 0, computed: expr } }
// $group - { $group: { _id: '$field', agg: { $sum: '$val' } } }
// $sort - { $sort: { field: 1 or -1 } }
// $limit - { $limit: N }
// $skip - { $skip: N }
// $lookup - { $lookup: { from, localField, foreignField, as } }
// $unwind - { $unwind: '$arrayField' }Pipeline Optimization: Stage Order Matters
MongoDB's query optimizer automatically reorders some pipeline stages for efficiency—for example, moving $match before $sort and $group when possible. However, you should always place $match as early as possible yourself: early filtering reduces the number of documents subsequent stages must process, and if $match is the first stage, MongoDB can use an index for it.
// BAD: expensive group before filter
db.orders.aggregate([
{ $group: { _id: '$userId', total: { $sum: '$amount' } } },
{ $match: { total: { $gt: 1000 } } } // too late to use index
]);
// GOOD: filter first, then group
db.orders.aggregate([
{ $match: { status: 'completed', createdAt: { $gte: thisYear } } },
{ $group: { _id: '$userId', total: { $sum: '$amount' } } },
{ $match: { total: { $gt: 1000 } } }
]);Arithmetic Expression Operators
Arithmetic expressions let you compute new values from existing fields within a stage. The most common ones are: $add, $subtract, $multiply, $divide, $mod, and $abs. These operators work on field references, literals, or the result of other expressions—enabling complex calculations that previously required application-side processing.
db.invoices.aggregate([
{ $project: {
subtotal: '$subtotal',
taxAmount: { $multiply: ['$subtotal', 0.08] }, // 8% tax
total: { $add: ['$subtotal', { $multiply: ['$subtotal', 0.08] }] },
discount: { $subtract: ['$listPrice', '$salePrice'] },
margin: { $divide: [{ $subtract: ['$price', '$cost'] }, '$price'] }
}}
]);Conditional Expressions: $cond and $ifNull
Conditional expressions allow if/else logic inside aggregation stages. $cond evaluates a boolean expression and returns one of two values (like a ternary operator). $ifNull returns a fallback value when a field is null or missing. These are essential for handling optional fields and computing derived categories without modifying the stored data.
db.products.aggregate([
{ $project: {
name: 1,
price: 1,
// Ternary: label products as budget or premium
tier: {
$cond: {
if: { $lt: ['$price', 100] },
then: 'budget',
else: 'premium'
}
},
// Fallback for missing description
description: { $ifNull: ['$description', 'No description available'] }
}}
]);String Expression Operators
String operators let you manipulate text fields within the pipeline: $concat joins strings, $toUpper/$toLower change case, $trim/$ltrim/$rtrim strip whitespace, $substr extracts a substring, and $split splits on a delimiter. These are useful for normalising data during aggregation without needing to update the stored documents.
db.users.aggregate([
{ $project: {
// Combine first and last name
fullName: { $concat: ['$firstName', ' ', '$lastName'] },
// Normalise email to lowercase
emailNorm: { $toLower: '$email' },
// Extract domain from email
domain: {
$arrayElemAt: [{ $split: ['$email', '@'] }, 1]
}
}}
]);Date Expression Operators
Date operators extract components from date fields for grouping and computation: $year, $month, $dayOfMonth, $hour, $minute, $dayOfWeek, and $dateToString. These are indispensable for time-series analytics like 'total orders per month' or 'average revenue by day of week'.
// Group orders by year and month
db.orders.aggregate([
{ $group: {
_id: {
year: { $year: '$createdAt' },
month: { $month: '$createdAt' }
},
count: { $sum: 1 },
revenue: { $sum: '$amount' }
}},
{ $sort: { '_id.year': 1, '_id.month': 1 } }
]);Quick Check
Test your understanding of aggregation pipeline concepts from this lesson.
Lesson Recap
In this lesson you learned: pipeline stages transform a stream of documents sequentially, stage operators define what a stage does while expression operators compute values within stages, and $ prefixes field references while $$ prefixes system variables. Next up we focus on $match and $project, the filter and reshape workhorses.
Frequently asked questions
Is the “Pipeline Concepts: Stages, Operators, and Expressions” lesson free?
Yes — the full text of “Pipeline Concepts: Stages, Operators, and Expressions” is free to read here on the web, and the MongoDB Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MongoDB Academy course, upgrade to CoddyKit PRO.
What will I learn in “Pipeline Concepts: Stages, Operators, and Expressions”?
Learners will understand how data flows through pipeline stages and distinguish between stage operators and expression operators. You practise MongoDB Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MongoDB Academy?
No prior experience is required. MongoDB Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Pipeline Concepts: Stages, Operators, and Expressions” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MongoDB Academy lesson?
Yes. Every MongoDB Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Pipeline Concepts: Stages, Operators, and Expressions
- $match and $project: Filter and Reshape
- $group: Aggregating and Computing Totals
- $sort, $limit, and $skip in the Pipeline