When the World Changes Under Your Model
A first look at data drift and concept drift.
When the World Changes Under Your Model is a free MLOps Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MLOps Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Models Get Stale
Your model is a snapshot of the past. As the real world moves on, that snapshot slowly goes out of date. This slow rot is called drift. 🌍
Data Drift
Data drift means the inputs change shape, like users suddenly being younger or shopping at new hours, even if the right answer rules stay the same.
Concept Drift
Concept drift is deeper: the relationship between inputs and the target changes. The same input now deserves a different prediction than before.
A Real Example
A spam filter faces concept drift constantly. Spammers change tactics, so yesterday's clear spam signal becomes today's normal email. The concept moved.
Sudden vs Gradual
Drift can be sudden, like a lockdown changing behavior overnight, or gradual, creeping over months. Both erode accuracy, just at different speeds.
Seasonal Patterns
Not every shift is permanent. Seasonal changes, like holiday shopping spikes, repeat each year and should not be mistaken for true drift.
Spotting Data Drift
To detect data drift, compare a feature's recent distribution against the training distribution and measure the gap between them.
ref = train['age']
cur = live['age']
# compare distributions, not single valuesSpotting Concept Drift
Concept drift is harder because you need real labels. Watch your accuracy over time, and a steady decline hints the concept has shifted.
Drift Is Not a Bug
Drift is not broken code, it is the world doing its job. The only real cure is keeping your model fresh through retraining.
Monitor, Then React
You cannot fix drift you never see. Set up drift monitoring first, then decide whether to alert, retrain, or roll back.
Two Names, One Goal
Remember the split: data drift is inputs changing, concept drift is the answer rule changing. Both signal it may be time to refresh the model.
Quick Check
Tell data drift and concept drift apart.
Recap
The world shifts under every model. Data drift changes inputs, concept drift changes the answer rule. Monitor both and retrain to stay accurate. ✅
Frequently asked questions
Is the “When the World Changes Under Your Model” lesson free?
Yes — the full text of “When the World Changes Under Your Model” is free to read here on the web, and the MLOps Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MLOps Academy course, upgrade to CoddyKit PRO.
What will I learn in “When the World Changes Under Your Model”?
A first look at data drift and concept drift. You practise MLOps Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MLOps Academy?
No prior experience is required. MLOps Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “When the World Changes Under Your Model” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MLOps Academy lesson?
Yes. Every MLOps Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The Training-Serving Skew Trap
- Silent Failures: No Crash, Wrong Answers
- When the World Changes Under Your Model
- The Reproducibility Problem