데이터와 모델 드리프트 탐지

Your model learned yesterday's patterns, but real data keeps changing. When it shifts, predictions quietly get worse. We call this problem drift. 🌊

There are two kinds to watch. Data drift means the inputs change shape, while concept drift means the relationship between inputs and labels changes.

A spam filter trained on old emails sees new tricks it never met. The inputs drifted, so its once-sharp accuracy slowly decays in production.

The first signal of drift is in the features. Compare the distribution of recent inputs to your training data and look for a shift in the shape.

To quantify how far two distributions diverge, use a statistic like KS, the Kolmogorov-Smirnov test, which returns a number plus a p-value.

A small p-value means the live data no longer matches training. Pick a threshold and trigger an alert whenever the gap crosses it.

Even without labels, the spread of your model's outputs is a clue. A sudden shift in predicted probabilities often signals incoming drift.

When real labels eventually arrive, compare them to past predictions. Falling live accuracy is the clearest, most direct proof that drift is hurting you.

You need not build everything by hand. Libraries like Evidently compute drift reports across all features and flag the columns that moved.

Drift is not a one-time check. Run these comparisons on a schedule so your monitoring catches slow shifts long before users complain.

Spotting drift early lets you act before damage spreads. The natural next move is to retrain on fresh data and restore the model's edge.

You learned to spot drift: compare live inputs to training, measure the gap with tests, watch predictions, and monitor on a schedule. 🎉