MongoDB Academy · درس

ما هو Atlas Data Federation؟

سيصف المتعلمون بنية Data Federation وأنواع مصادر البيانات التي تدعمها ومحرك الاستعلام الذي يوحّد بينها.

ما هو Atlas Data Federation؟ درس مجاني في MongoDB Academy على CoddyKit. هذا هو الدرس 1 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في MongoDB Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة MongoDB Academy 4 دروس في المجموع.

بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.

The Problem: Data Lives Everywhere

Modern applications generate data across multiple systems: live operational data in MongoDB Atlas, historical archives in Amazon S3, and analytics exports in data lakes. Querying across these silos traditionally requires data pipelines, ETL jobs, and separate query engines. Atlas Data Federation solves this by letting you query all these sources with a single MongoDB connection and the familiar aggregation pipeline.

What Is Atlas Data Federation?

Atlas Data Federation is a fully managed query engine built into MongoDB Atlas. It creates a federated database instance — a virtual MongoDB deployment that maps data from multiple sources (Atlas clusters, S3 buckets, Atlas Data Lake, HTTP endpoints) to virtual collections. You connect with a standard MongoDB connection string and use the same aggregation pipeline you already know.

// Connect to a federated database instance
// The URI looks like a regular Atlas connection string
// mongodb://...@data.mongodb-api.com/federated
const client = new MongoClient(
  'mongodb+srv://federated-instance.mongodb.net/myFederatedDB'
)

Supported Data Sources

Atlas Data Federation can query: Atlas clusters — live MongoDB collections. Amazon S3 — JSON, BSON, CSV, TSV, Avro, ORC, and Parquet files stored in S3 buckets. Atlas Data Lake — processed and enriched datasets. HTTP/HTTPS endpoints — external REST APIs that return JSON. All sources are mapped to virtual namespaces within the federated instance.

The Federated Database Architecture

A federated database has three layers: Storage configuration — defines which sources map to which virtual databases and collections. Query engine — the distributed SQL/MQL processor that reads from multiple sources, applies pipeline stages, and merges results. Connection layer — a mongos-compatible endpoint you connect to with any MongoDB driver or mongosh.

// Storage configuration (simplified JSON structure)
{
  'databases': [{
    'name': 'analytics',
    'collections': [{
      'name': 'orders_archive',
      'dataSources': [{
        'storeName': 's3Store',
        'path': '/data/orders/2024/'
      }]
    }]
  }]
}

Creating a Federated Database Instance

You create a federated database instance through the Atlas UI, Atlas Admin API, or Atlas CLI. During setup you: 1) Name the instance. 2) Add stores (S3 buckets with IAM credentials, Atlas clusters, etc.). 3) Define virtual databases and collections that point to those stores. 4) Copy the connection string and connect with your MongoDB driver.

// Using Atlas CLI to create a data federation instance
// atlas dataFederation create myFederation --region US_EAST_1

// Then add a store via Atlas UI or API
// POST /api/atlas/v1.0/groups/{groupId}/dataFederation/{name}/dataStores
// { 'name': 's3Store', 'provider': 'S3', 'region': 'us-east-1', 'bucket': 'my-data' }

Virtual Namespaces: Collections Without Schemas

Virtual collections in a federated database do not store data — they are logical views over the underlying source files or collections. You can query a virtual collection named analytics.orders that actually reads S3 Parquet files at s3://my-bucket/orders/2024/. To MongoDB drivers and tools, the virtual collection looks and behaves like a regular MongoDB collection.

// Query a virtual collection backed by S3 files
const ordersArchive = db.collection('orders_archive')
const result = await ordersArchive.aggregate([
  { $match: { year: 2024, region: 'EU' } },
  { $group: { _id: '$category', total: { $sum: '$revenue' } } },
  { $sort: { total: -1 } }
]).toArray()

Cross-Source Joins With $lookup

One of the most powerful features is joining a live Atlas collection with archived S3 data in a single pipeline. For example: look up active customer details from a live Atlas cluster and join them with their 3-year purchase history stored in S3 Parquet files — all in one aggregation with no ETL job required.

// Join live Atlas collection with S3 archive
db.customers.aggregate([
  { $match: { tier: 'platinum' } },      // live Atlas
  { $lookup: {
    from: 'orders_archive',               // virtual S3-backed collection
    localField: '_id',
    foreignField: 'customerId',
    as: 'purchaseHistory'
  }},
  { $project: { name: 1, tier: 1,
    totalOrders: { $size: '$purchaseHistory' } } }
])

File Format Support in S3

Data Federation reads S3 files in many formats: JSON (one document per line or array), BSON (MongoDB native binary), CSV/TSV (with header row), Avro, ORC, and Parquet (columnar formats widely used in data lakes). For columnar formats, Data Federation can push projection and filter predicates into the file reader for even faster scans.

// In storage config, specify file format per path
{
  'dataSources': [{
    'storeName': 's3Store',
    'path': '/analytics/events/{year string}/{month string}/',
    'defaultFormat': '.parquet'
  }]
}

Cost Model: Query-Based Pricing

Atlas Data Federation charges based on data processed (bytes scanned), not uptime. This makes it cost-effective for infrequent analytical queries over large S3 archives — you pay nothing when no queries run. However, scanning entire unpartitioned S3 datasets can become expensive. Partitioning your S3 data and using projection to reduce scanned bytes are critical for cost control.

Security: Auth and Network

Federated database instances use the same Atlas database users and roles as regular Atlas clusters. You can apply Atlas network peering, private endpoints (AWS PrivateLink), and IP Access Lists to restrict who can connect. The connection to S3 uses IAM roles rather than storing AWS keys directly, following AWS security best practices.

When to Use Atlas Data Federation

Data Federation is a good fit when: 1) You need to run ad-hoc queries across historical S3 archives without loading data into a live cluster. 2) You want to join live transactional data with archived data in a single query. 3) You need a unified analytics interface across multiple Atlas clusters. 4) You want to avoid building and maintaining a separate ETL pipeline for each analytical use case.

Quick Check

Test your understanding of MongoDB & NoSQL Databases concepts from this lesson.

Lesson Recap

In this lesson you learned: Atlas Data Federation creates a virtual MongoDB namespace over S3, Atlas clusters, and other sources, you use the standard aggregation pipeline to query and join data across all sources in one operation, and costs are based on bytes scanned, so partitioning and projection are essential for cost control. Next up we map S3 and Atlas sources to virtual namespaces.

الأسئلة الشائعة

هل درس «ما هو Atlas Data Federation؟» مجاني؟

نعم — نص درس «ما هو Atlas Data Federation؟» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة MongoDB Academy، انتقل إلى CoddyKit PRO. تتضمن دورة MongoDB Academy 4 دروس في المجموع.

ماذا ستتعلم في «ما هو Atlas Data Federation؟»؟

سيصف المتعلمون بنية Data Federation وأنواع مصادر البيانات التي تدعمها ومحرك الاستعلام الذي يوحّد بينها. تتمرن على MongoDB Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.

هل أحتاج إلى خبرة سابقة لأبدأ MongoDB Academy؟

لا تُشترط خبرة سابقة. MongoDB Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 1 من أصل 4.

كم من الوقت يستغرق درس «ما هو Atlas Data Federation؟»؟

معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.

هل يمكنني كتابة وتشغيل أكواد في درس MongoDB Academy هذا؟

نعم. كل درس في MongoDB Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.

جميع الدروس في هذه الدورة

← العودة إلى MongoDB Academy