Dynamic vs. Explicit Mappings

When you put data into Elasticsearch, it needs to know what kind of data each field holds. This definition is called a mapping.

Think of a mapping as the schema for your documents, similar to a table schema in a traditional database.

There are two main ways Elasticsearch handles mappings: dynamic (automatic) and explicit (manual).

Dynamic mapping is Elasticsearch's default behavior. It's like an intelligent assistant that tries to guess the data type of new fields automatically.

When you index a document with a new field that Elasticsearch hasn't seen before, it analyzes the field's value and assigns a default mapping to it.

The magic happens with the first document containing a new field. Elasticsearch inspects the value and infers its type:

• "Hello" → text and keyword
• 123 → long
• true → boolean
• "2023-01-01" → date

This makes getting started very quick!

Dynamic mapping offers several advantages:

• Ease of Use: No need to pre-define your schema. Just index documents!
• Flexibility: Easily adapt to changing data structures or add new fields without modifying existing mappings.
• Rapid Prototyping: Great for initial data exploration and quick development cycles.

While convenient, dynamic mapping has downsides, especially in production:

• Inconsistent Types: A field might be mapped as a long, then later you index a string, leading to errors.
• Performance Overhead: Inferring mappings takes resources. Too many unique fields can strain the cluster.
• Schema Drift: Unintended fields can be indexed, cluttering your schema and potentially causing issues.

Explicit mapping means you manually define the schema for your index before you add any documents.

You tell Elasticsearch exactly what type each field should be, how it should be analyzed, and other specific settings.

For production systems, explicit mapping is usually preferred:

• Data Consistency: Guarantees fields always have the correct type.
• Optimized Performance: Knowing field types upfront allows Elasticsearch to store and search data more efficiently.
• Error Prevention: Prevents unexpected data types or schema changes from breaking your application.
• Full Control: Fine-tune every aspect of how your data is handled.

You can control how Elasticsearch handles new fields using the dynamic setting within your mapping. It can be set to:

• true (default): New fields are added dynamically.
• false: New fields are completely ignored.
• strict: New fields throw an error, preventing indexing.

This setting can be applied at the index level or for specific object fields.

Setting "dynamic": "false" tells Elasticsearch to completely ignore any new fields that appear in documents. They won't be indexed or searchable.

This is useful if you want to prevent accidental schema changes but don't want to fail document indexing.

When "dynamic": "strict", any document containing a field not explicitly defined in the mapping will cause an indexing error.

This is the strictest approach, ensuring your schema is always exactly what you defined.

Consider an index with the following mapping:

PUT /my_data
{
  "mappings": {
    "dynamic": "false",
    "properties": {
      "id": { "type": "keyword" },
      "value": { "type": "long" }
    }
  }
}

What happens if you try to index the following document?

PUT /my_data/_doc/1
{
  "id": "A1",
  "value": 100,
  "new_field": "extra data"
}

You've learned about the two main approaches to mapping in Elasticsearch:

• Dynamic Mapping: Automatic, flexible, great for quick starts but can lead to inconsistencies.
• Explicit Mapping: Manual, controlled, crucial for production for data integrity and performance.

The dynamic setting (true, false, strict) gives you fine-grained control over how Elasticsearch reacts to new fields. Choose wisely based on your project's needs!