What Is Atlas Data Federation?
Learners will describe the Data Federation architecture, the types of data sources it supports, and the query engine that unifies them.
What Is Atlas Data Federation? is a free MongoDB Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MongoDB Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Problem: Data Lives Everywhere
Modern applications generate data across multiple systems: live operational data in MongoDB Atlas, historical archives in Amazon S3, and analytics exports in data lakes. Querying across these silos traditionally requires data pipelines, ETL jobs, and separate query engines. Atlas Data Federation solves this by letting you query all these sources with a single MongoDB connection and the familiar aggregation pipeline.
What Is Atlas Data Federation?
Atlas Data Federation is a fully managed query engine built into MongoDB Atlas. It creates a federated database instance — a virtual MongoDB deployment that maps data from multiple sources (Atlas clusters, S3 buckets, Atlas Data Lake, HTTP endpoints) to virtual collections. You connect with a standard MongoDB connection string and use the same aggregation pipeline you already know.
// Connect to a federated database instance
// The URI looks like a regular Atlas connection string
// mongodb://...@data.mongodb-api.com/federated
const client = new MongoClient(
'mongodb+srv://federated-instance.mongodb.net/myFederatedDB'
)Supported Data Sources
Atlas Data Federation can query: Atlas clusters — live MongoDB collections. Amazon S3 — JSON, BSON, CSV, TSV, Avro, ORC, and Parquet files stored in S3 buckets. Atlas Data Lake — processed and enriched datasets. HTTP/HTTPS endpoints — external REST APIs that return JSON. All sources are mapped to virtual namespaces within the federated instance.
The Federated Database Architecture
A federated database has three layers: Storage configuration — defines which sources map to which virtual databases and collections. Query engine — the distributed SQL/MQL processor that reads from multiple sources, applies pipeline stages, and merges results. Connection layer — a mongos-compatible endpoint you connect to with any MongoDB driver or mongosh.
// Storage configuration (simplified JSON structure)
{
'databases': [{
'name': 'analytics',
'collections': [{
'name': 'orders_archive',
'dataSources': [{
'storeName': 's3Store',
'path': '/data/orders/2024/'
}]
}]
}]
}Creating a Federated Database Instance
You create a federated database instance through the Atlas UI, Atlas Admin API, or Atlas CLI. During setup you: 1) Name the instance. 2) Add stores (S3 buckets with IAM credentials, Atlas clusters, etc.). 3) Define virtual databases and collections that point to those stores. 4) Copy the connection string and connect with your MongoDB driver.
// Using Atlas CLI to create a data federation instance
// atlas dataFederation create myFederation --region US_EAST_1
// Then add a store via Atlas UI or API
// POST /api/atlas/v1.0/groups/{groupId}/dataFederation/{name}/dataStores
// { 'name': 's3Store', 'provider': 'S3', 'region': 'us-east-1', 'bucket': 'my-data' }Virtual Namespaces: Collections Without Schemas
Virtual collections in a federated database do not store data — they are logical views over the underlying source files or collections. You can query a virtual collection named analytics.orders that actually reads S3 Parquet files at s3://my-bucket/orders/2024/. To MongoDB drivers and tools, the virtual collection looks and behaves like a regular MongoDB collection.
// Query a virtual collection backed by S3 files
const ordersArchive = db.collection('orders_archive')
const result = await ordersArchive.aggregate([
{ $match: { year: 2024, region: 'EU' } },
{ $group: { _id: '$category', total: { $sum: '$revenue' } } },
{ $sort: { total: -1 } }
]).toArray()Cross-Source Joins With $lookup
One of the most powerful features is joining a live Atlas collection with archived S3 data in a single pipeline. For example: look up active customer details from a live Atlas cluster and join them with their 3-year purchase history stored in S3 Parquet files — all in one aggregation with no ETL job required.
// Join live Atlas collection with S3 archive
db.customers.aggregate([
{ $match: { tier: 'platinum' } }, // live Atlas
{ $lookup: {
from: 'orders_archive', // virtual S3-backed collection
localField: '_id',
foreignField: 'customerId',
as: 'purchaseHistory'
}},
{ $project: { name: 1, tier: 1,
totalOrders: { $size: '$purchaseHistory' } } }
])File Format Support in S3
Data Federation reads S3 files in many formats: JSON (one document per line or array), BSON (MongoDB native binary), CSV/TSV (with header row), Avro, ORC, and Parquet (columnar formats widely used in data lakes). For columnar formats, Data Federation can push projection and filter predicates into the file reader for even faster scans.
// In storage config, specify file format per path
{
'dataSources': [{
'storeName': 's3Store',
'path': '/analytics/events/{year string}/{month string}/',
'defaultFormat': '.parquet'
}]
}Cost Model: Query-Based Pricing
Atlas Data Federation charges based on data processed (bytes scanned), not uptime. This makes it cost-effective for infrequent analytical queries over large S3 archives — you pay nothing when no queries run. However, scanning entire unpartitioned S3 datasets can become expensive. Partitioning your S3 data and using projection to reduce scanned bytes are critical for cost control.
Security: Auth and Network
Federated database instances use the same Atlas database users and roles as regular Atlas clusters. You can apply Atlas network peering, private endpoints (AWS PrivateLink), and IP Access Lists to restrict who can connect. The connection to S3 uses IAM roles rather than storing AWS keys directly, following AWS security best practices.
When to Use Atlas Data Federation
Data Federation is a good fit when: 1) You need to run ad-hoc queries across historical S3 archives without loading data into a live cluster. 2) You want to join live transactional data with archived data in a single query. 3) You need a unified analytics interface across multiple Atlas clusters. 4) You want to avoid building and maintaining a separate ETL pipeline for each analytical use case.
Quick Check
Test your understanding of MongoDB & NoSQL Databases concepts from this lesson.
Lesson Recap
In this lesson you learned: Atlas Data Federation creates a virtual MongoDB namespace over S3, Atlas clusters, and other sources, you use the standard aggregation pipeline to query and join data across all sources in one operation, and costs are based on bytes scanned, so partitioning and projection are essential for cost control. Next up we map S3 and Atlas sources to virtual namespaces.
Frequently asked questions
Is the “What Is Atlas Data Federation?” lesson free?
Yes — the full text of “What Is Atlas Data Federation?” is free to read here on the web, and the MongoDB Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MongoDB Academy course, upgrade to CoddyKit PRO.
What will I learn in “What Is Atlas Data Federation?”?
Learners will describe the Data Federation architecture, the types of data sources it supports, and the query engine that unifies them. You practise MongoDB Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MongoDB Academy?
No prior experience is required. MongoDB Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “What Is Atlas Data Federation?” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MongoDB Academy lesson?
Yes. Every MongoDB Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.