概念漂移：X 与 Y 之间的关系发生变化
您将使用时间序列示例区分数据漂移与概念漂移，并理解为什么数据漂移并不总是意味着模型性能下降。
概念漂移：X 与 Y 之间的关系发生变化 是 CoddyKit 上的免费 Machine Learning Academy 课时。 这是第 2 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
What Is Concept Drift?
Concept drift occurs when the relationship between input features X and the target label Y changes over time — even if the feature distributions themselves remain stable. In fraud detection, fraudsters learn which transactions are flagged and change their patterns, so transactions that looked fraudulent in 2022 look benign by 2024. The 'concept' being learned (what fraud looks like) has drifted, but the feature distributions may appear unchanged, making concept drift harder to detect than data drift.
Data Drift vs Concept Drift: Key Difference
The critical distinction: data drift is a change in P(X) — the input feature distribution. Concept drift is a change in P(Y|X) — the conditional relationship between features and labels. Data drift does not always degrade model performance (if the new distribution is still within the learned decision boundary). Concept drift always degrades performance because the model's learned mapping from X to Y is now incorrect, regardless of whether X looks the same.
import numpy as np
import pandas as pd
# Scenario: credit scoring model
# Feature: income. Label: 1 = defaults, 0 = repays
# Training period: incomes 30-50k correlate with defaults
np.random.seed(42)
train_income = np.random.normal(40000, 8000, 1000)
train_default = (train_income < 35000).astype(int) # low income -> default
# Concept drift: inflation shifts threshold; now defaults start at 50k
prod_income = np.random.normal(40000, 8000, 500) # SAME distribution (no data drift)
prod_default = (prod_income < 50000).astype(int) # new relationship
print('Train default rate:', train_default.mean().round(3))
print('Prod default rate:', prod_default.mean().round(3))
print('Feature distribution same (no data drift).',
'But Y|X relationship has changed (concept drift).')Types of Concept Drift
Concept drift comes in four varieties. Sudden drift: the relationship changes abruptly (e.g., COVID-19 changing consumer spending patterns overnight). Gradual drift: the old concept slowly fades and a new concept emerges (e.g., shifting definition of spam over months). Incremental drift: a continuous, slow drift with no clear breakpoint. Recurring drift: seasonal patterns — the relationship at Christmas differs from July, but July eventually returns.
import numpy as np
import matplotlib.pyplot as plt
t = np.linspace(0, 100, 1000)
# Sudden drift: step function at t=50
sudden = np.where(t < 50, 0.8, 0.3) # accuracy drops at t=50
# Gradual drift: linear transition
gradual = np.where(t < 40, 0.8,
np.where(t > 60, 0.4,
0.8 - (t - 40) * 0.02))
# Recurring drift: seasonal pattern
recurring = 0.6 + 0.2 * np.sin(2 * np.pi * t / 20)
for name, series in [('Sudden', sudden), ('Gradual', gradual), ('Recurring', recurring)]:
print(f'{name} drift -- min acc: {series.min():.2f}, max: {series.max():.2f}')Detecting Concept Drift via Performance Monitoring
The most reliable signal for concept drift is declining model performance on labelled production data. Track your chosen metric (accuracy, F1, AUC) on a rolling window of recent predictions where ground-truth labels have become available. A downward trend in performance that is not explained by data drift (the features look the same) is strong evidence of concept drift. This requires a feedback loop: collecting true labels for production predictions, which can take days to months depending on the task.
import numpy as np
import pandas as pd
np.random.seed(42)
# Simulate weekly model accuracy tracking
weekly_accuracy = [
0.92, 0.91, 0.90, 0.89, 0.88, # slight decline
0.86, 0.83, 0.79, 0.74, 0.68, # accelerating decline = concept drift
0.65, 0.61
]
df = pd.DataFrame({'week': range(1, 13), 'accuracy': weekly_accuracy})
df['rolling_mean'] = df['accuracy'].rolling(3).mean()
df['alert'] = df['accuracy'] < 0.75 # threshold
print(df.to_string(index=False))
print('Weeks with alerts:', df[df['alert']]['week'].tolist())Detecting Concept Drift Without Labels
Waiting for ground-truth labels is slow. When labels arrive with high latency, use prediction distribution monitoring as an early warning signal. If P(X) is stable (no data drift) but the model's predicted label distribution shifts significantly, this suggests the underlying concept has changed. For example, if a binary classifier's positive-prediction rate drops from 30% to 5% without any change in feature distributions, the model is likely seeing a different relationship between features and the positive class.
import numpy as np
# Monitoring prediction distributions as a proxy for concept drift
# Period 1 (training distribution): ~30% positive predictions
period1_probs = np.random.beta(2, 5, 1000) # right-skewed, ~30% above 0.5
period1_pos_rate = (period1_probs > 0.5).mean()
# Period 2 (concept drifted): only ~5% positive predictions (model confused)
period2_probs = np.random.beta(1, 15, 500) # very right-skewed
period2_pos_rate = (period2_probs > 0.5).mean()
print(f'Period 1 positive rate: {period1_pos_rate:.2%}')
print(f'Period 2 positive rate: {period2_pos_rate:.2%}')
print(f'Rate drop: {(period1_pos_rate - period2_pos_rate):.2%}')
print('Possible concept drift detected!')Windowed Retraining: Adapting to Concept Drift
The most common strategy for handling concept drift is windowed retraining: periodically retrain the model using only the most recent N examples, discarding old data that reflects outdated patterns. A sliding window keeps the most recent N examples; an expanding window uses all data with recency weighting. The optimal window size depends on how fast the concept evolves — fast-changing domains (financial markets) need smaller windows than slow-changing ones (weather forecasting).
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
np.random.seed(42)
def simulate_concept_drift(n, t):
X = np.random.normal(0, 1, (n, 2))
# Decision boundary rotates over time
y = ((X[:, 0] * np.cos(t) - X[:, 1] * np.sin(t)) > 0).astype(int)
return X, y
# Full retrain vs sliding window retrain
for window_size in [100, 500, None]: # None = all data
all_X, all_y = [], []
accs = []
for t in np.linspace(0, np.pi, 10):
X, y = simulate_concept_drift(200, t)
all_X.append(X)
all_y.append(y)
if window_size:
train_X = np.vstack(all_X[-window_size//200:])
train_y = np.hstack(all_y[-window_size//200:])
else:
train_X = np.vstack(all_X)
train_y = np.hstack(all_y)
clf = LogisticRegression().fit(train_X, train_y)
accs.append(clf.score(X, y))
print(f'Window={window_size}: avg accuracy={np.mean(accs):.3f}')Sample Weighting: Downweighting Old Data
Instead of a hard window cutoff, assign exponentially decaying weights to training examples so recent examples contribute more to the model. Examples from last week get weight 1.0; examples from last month get weight 0.5; examples from six months ago get weight 0.1. Many scikit-learn estimators accept a sample_weight parameter in their fit method, making this easy to implement without discarding any data.
import numpy as np
from sklearn.ensemble import GradientBoostingClassifier
np.random.seed(42)
# Simulate 1000 training examples collected over 10 weeks
X = np.random.normal(0, 1, (1000, 5))
y = (X[:, 0] + np.random.normal(0, 0.5, 1000) > 0).astype(int)
weeks_ago = np.linspace(10, 0, 1000) # example 0 is oldest, 999 is newest
# Exponential decay: half-life of 3 weeks
half_life = 3.0
sample_weights = np.exp(-weeks_ago * np.log(2) / half_life)
sample_weights /= sample_weights.sum()
clf = GradientBoostingClassifier(n_estimators=100, random_state=42)
clf.fit(X, y, sample_weight=sample_weights * len(y)) # scale up
print('Model trained with recency-weighted samples.')
print('Weight ratio new/old:', round(sample_weights[-1] / sample_weights[0], 2))CUSUM and ADWIN Drift Detectors
Dedicated concept drift detection algorithms provide more principled alerts. CUSUM (Cumulative Sum) detects changes in the mean of a sequence by accumulating deviations from a reference value. ADWIN (Adaptive Windowing) maintains a variable-length window and shrinks it when a statistical test detects a distribution change within the window. The river library (formerly scikit-multiflow) implements both for streaming data.
# Using the river library for streaming drift detection
# pip install river
# from river.drift import ADWIN
# adwin = ADWIN(delta=0.002) # delta controls sensitivity
# error_sequence = [0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1] # errors over time
#
# for i, error in enumerate(error_sequence):
# adwin.update(error)
# if adwin.drift_detected:
# print(f'ADWIN detected drift at step {i}')
# adwin = ADWIN(delta=0.002) # reset detector
# Manual CUSUM example:
def cusum(errors, threshold=5, drift=0.01):
cum_sum, alerts = 0, []
for i, e in enumerate(errors):
cum_sum = max(0, cum_sum + e - drift)
if cum_sum > threshold:
alerts.append(i)
cum_sum = 0
return alerts
errors = [0]*10 + [1]*15 # errors spike after step 10
print('CUSUM drift alerts at steps:', cusum(errors))Concept Drift vs Seasonality
Seasonality is predictable periodic variation (Christmas shopping spikes, summer travel patterns) while concept drift is an unpredictable one-directional shift. A good monitoring system distinguishes them by checking historical seasonality patterns. If the current distribution matches last year's same-period distribution, it is likely seasonality rather than drift. Include time-of-year features in your model to handle predictable seasonality rather than triggering unnecessary retraining.
import numpy as np
from scipy.stats import ks_2samp
np.random.seed(42)
# Training: July 2023 data
july_2023 = np.random.normal(100, 20, 1000) # low activity (summer)
# Monitoring: July 2024 (seasonal -- same month last year)
july_2024 = np.random.normal(102, 21, 500) # similar to last July
# Monitoring: December 2024 (seasonal spike -- same as Dec 2023)
dec_2024 = np.random.normal(180, 30, 500) # high activity (Christmas)
stat_july, p_july = ks_2samp(july_2023, july_2024)
stat_dec, p_dec = ks_2samp(july_2023, dec_2024)
print(f'July 2024 vs July 2023: KS={stat_july:.3f} p={p_july:.3f} -> {"drift" if p_july < 0.05 else "seasonal OK"}')
print(f'Dec 2024 vs July 2023: KS={stat_dec:.3f} p={p_dec:.3f} -> Expected seasonal shift')Building a Concept Drift Response Plan
Respond to concept drift with a tiered plan. Tier 1 (early warning): alert on prediction distribution shifts without waiting for labels. Tier 2 (confirmation): when labels arrive, confirm performance degradation exceeds 5% relative. Tier 3 (action): trigger automatic windowed retraining using the drift detection timestamp as the window cutoff. Tier 4 (escalation): if retraining does not recover performance, escalate to a data scientist for feature engineering review.
def concept_drift_response(current_accuracy, baseline_accuracy,
data_drift_detected, prediction_rate_shift):
relative_drop = (baseline_accuracy - current_accuracy) / baseline_accuracy
if relative_drop > 0.10:
return ('TIER 4 ESCALATION: >10% accuracy drop. '
'Manual feature engineering review required.')
elif relative_drop > 0.05:
return ('TIER 3 ACTION: 5-10% accuracy drop. '
'Trigger windowed retraining immediately.')
elif data_drift_detected or prediction_rate_shift > 0.15:
return ('TIER 2 MONITOR: Early warning signals present. '
'Increase monitoring to daily. Queue retraining.')
else:
return 'TIER 1 OK: No significant concept drift detected.'
print(concept_drift_response(0.84, 0.92, False, 0.05)) # TIER 3
print(concept_drift_response(0.91, 0.92, True, 0.18)) # TIER 2
print(concept_drift_response(0.91, 0.92, False, 0.03)) # TIER 1Ensemble Models and Concept Drift Resilience
Ensemble methods like random forests and gradient boosting can mask concept drift for longer than single models because their diversity makes them less sensitive to any one feature distribution shift. However, this resilience is a double-edged sword: the model degrades more slowly and more silently. Monitor the individual tree disagreement rate in a forest — when trees increasingly disagree on predictions, it is an early signal that the concept is drifting and the ensemble is struggling to reach consensus.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
# Measure inter-tree disagreement on current data as drift signal
def inter_tree_disagreement(forest, X):
predictions = np.array([tree.predict(X) for tree in forest.estimators_])
# Fraction of pairs of trees that disagree per sample, averaged over samples
n_trees = len(forest.estimators_)
disagreement_per_sample = []
for i in range(X.shape[0]):
preds = predictions[:, i]
disagree = (preds != preds[0]).sum() / n_trees
disagreement_per_sample.append(disagree)
return np.mean(disagreement_per_sample)
# Higher disagreement = more concept drift
# print(inter_tree_disagreement(model, X_current))Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: concept drift is a change in P(Y|X) — the mapping from features to labels — which can occur even when feature distributions appear stable, declining model performance on labelled production data is the definitive signal of concept drift, and windowed retraining and sample weighting are the primary adaptation strategies to keep models current with evolving concepts. Next up we look at monitoring prediction distributions and confidence scores as an early warning system when ground-truth labels are not yet available.
常见问题解答
「概念漂移：X 与 Y 之间的关系发生变化」课时是免费的吗？
是的 — 「概念漂移：X 与 Y 之间的关系发生变化」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。
「概念漂移：X 与 Y 之间的关系发生变化」这节课中我会学到什么？
您将使用时间序列示例区分数据漂移与概念漂移，并理解为什么数据漂移并不总是意味着模型性能下降。 你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Machine Learning Academy 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 2 节课，共 4 节。
「概念漂移：X 与 Y 之间的关系发生变化」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Machine Learning Academy 课中编写并运行代码吗？
能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。
此课程中的所有课时
- 数据漂移：特征分布随时间发生变化
- 概念漂移：X 与 Y 之间的关系发生变化
- 监控预测分布与置信度分数
- 使用 Evidently AI 构建数据漂移告警流水线