Index Lifecycle Management (ILM)
Master Index Lifecycle Management to automate rollover, the hot-warm-cold architecture, and retention so log indices stay fast and storage stays under control.
Index Lifecycle Management (ILM) is a free System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Manage Index Lifecycle?
Time-series data like logs grows endlessly. Index Lifecycle Management (ILM) automates moving, shrinking, and deleting indices so old data does not crush your cluster.
Rollover
Rollover creates a new index when the current one hits a size, age, or document count threshold. Writes always go to a single alias.
rollover:
max_size: 50gb
max_age: 1d
max_docs: 100000000Write Aliases
Apps write to an alias such as logs-write. ILM swaps which physical index the alias points to on rollover, invisible to the writer.
POST logs-write/_doc
{ "message": "..." }The Phases
ILM defines lifecycle phases a data ages through.
- Hot: actively written and queried
- Warm: read-only, queried less
- Cold: rarely queried, cheap storage
- Delete: removed
Hot Phase
The hot phase lives on the fastest nodes with the most resources, handling all writes and recent queries.
phases:
hot:
actions:
rollover: { max_age: 1d }Warm Phase
In warm, indices become read-only and may move to cheaper nodes. You can shrink shards and force-merge segments to save space.
warm:
min_age: 7d
actions:
forcemerge: { max_num_segments: 1 }
shrink: { number_of_shards: 1 }Cold and Frozen
Cold data moves to the cheapest disks; frozen can be searched directly from object storage. Both trade query speed for cost.
cold:
min_age: 30d
actions:
allocate: { require: { data: cold } }Delete Phase
The delete phase enforces retention by removing indices past a chosen age, the simplest way to cap storage growth.
delete:
min_age: 90d
actions:
delete: {}Attaching a Policy
An index template binds new indices to an ILM policy and the write alias automatically.
PUT _index_template/logs
{ "template": { "settings": {
"index.lifecycle.name": "logs-policy" } } }Node Roles
Hot-warm-cold relies on node attributes so allocation actions can route shards to the right hardware tier.
node.attr.data: hot # on fast nodes
node.attr.data: cold # on cheap nodesPutting It Together
A typical logging policy: roll over daily in hot, move to warm and force-merge after a week, go cold after a month, and delete after ninety days.
Quick Check
Pick the phase that enforces retention.
Recap
You learned how ILM automates index management: rollover starts fresh indices behind a write alias, and the hot-warm-cold-delete phases move data to cheaper tiers as it ages and finally remove it. Index templates bind policies automatically, keeping clusters fast and storage bounded.
Frequently asked questions
Is the “Index Lifecycle Management (ILM)” lesson free?
Yes — the full text of “Index Lifecycle Management (ILM)” is free to read here on the web, and the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) course, upgrade to CoddyKit PRO.
What will I learn in “Index Lifecycle Management (ILM)”?
Master Index Lifecycle Management to automate rollover, the hot-warm-cold architecture, and retention so log indices stay fast and storage stays under control. You practise System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry)?
No prior experience is required. System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Index Lifecycle Management (ILM)” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson?
Yes. Every System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Elasticsearch Query Language (DSL)
- Logstash Filters and Pipelines
- Kibana Discover and Lens
- Index Lifecycle Management (ILM)