Stages: Ingest, Prep, Train, Eval
Break training into clear, cacheable steps.
Stages: Ingest, Prep, Train, Eval is a free MLOps Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MLOps Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
From One Big Script to Stages
A single train.py that does everything is hard to reuse and slow to debug. Splitting it into stages makes each step clear and independently runnable. 🧩
The Four Classic Stages
Most training pipelines follow the same shape: ingest, prep, train, eval. Each stage takes inputs and produces outputs the next one reads.
Stage 1: Ingest
The ingest stage pulls raw data from its source, like a database or bucket, and writes it to a known local path. Nothing is cleaned yet.
import pandas as pd
df = pd.read_csv("s3://bucket/raw.csv")
df.to_parquet("data/raw.parquet")Stage 2: Prep
The prep stage cleans and transforms raw data into model-ready features. It handles missing values, encoding, and splitting into train and test sets.
Stage 3: Train
The train stage reads prepared features and fits the model, then saves the fitted artifact to disk for the next stage to score.
model.fit(X_train, y_train)
joblib.dump(model, "models/model.pkl")Stage 4: Eval
The eval stage loads the saved model, scores it on the held-out test set, and writes metrics like accuracy to a file you can track.
score = model.score(X_test, y_test)
json.dump({"accuracy": score}, open("metrics.json", "w"))Stages Pass Files, Not Variables
Stages talk to each other through files on disk, not in-memory variables. That decoupling is what lets you re-run one stage without the others.
Why Stages Are Cacheable
Because each stage has fixed inputs and outputs, a tool can cache its result. If the inputs did not change, the stage is skipped entirely.
Each Stage Is Its Own Script
Give every stage one small script with a clear job. ingest.py, prep.py, train.py, eval.py read inputs and write outputs, nothing more.
Stages Form a DAG
Stages connect into a DAG, a directed graph where each step depends on the ones before it. Eval depends on train, train on prep, prep on ingest.
Clear Stages Make Debugging Easy
When a run fails, distinct stages tell you exactly where it broke. You can re-run just the failed stage instead of the whole pipeline.
Quick Check
Let us check how stages communicate.
Recap: Ingest, Prep, Train, Eval
You learned to split training into four cacheable stages that pass data through files. Clear stages mean faster reruns and far easier debugging. 🚀
Frequently asked questions
Is the “Stages: Ingest, Prep, Train, Eval” lesson free?
Yes — the full text of “Stages: Ingest, Prep, Train, Eval” is free to read here on the web, and the MLOps Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MLOps Academy course, upgrade to CoddyKit PRO.
What will I learn in “Stages: Ingest, Prep, Train, Eval”?
Break training into clear, cacheable steps. You practise MLOps Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MLOps Academy?
No prior experience is required. MLOps Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Stages: Ingest, Prep, Train, Eval” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MLOps Academy lesson?
Yes. Every MLOps Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Stages: Ingest, Prep, Train, Eval
- Define a Pipeline with DVC Stages
- Cache and Skip Unchanged Steps
- Parameterize Runs with params.yaml