Deriva de conceito: quando a relação entre X e Y muda
Os alunos distinguirão a deriva dos dados da deriva de conceito usando um exemplo de série temporal e entenderão por que a deriva dos dados nem sempre implica degradação do desempenho do modelo.
Deriva de conceito: quando a relação entre X e Y muda é uma aula grátis de Machine Learning Academy no CoddyKit. Esta é a aula 2 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de Machine Learning Academy, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de Machine Learning Academy inclui 4 aulas no total.
Partes desta aula ainda não foram traduzidas e aparecem em inglês.
What Is Concept Drift?
Concept drift occurs when the relationship between input features X and the target label Y changes over time — even if the feature distributions themselves remain stable. In fraud detection, fraudsters learn which transactions are flagged and change their patterns, so transactions that looked fraudulent in 2022 look benign by 2024. The 'concept' being learned (what fraud looks like) has drifted, but the feature distributions may appear unchanged, making concept drift harder to detect than data drift.
Data Drift vs Concept Drift: Key Difference
The critical distinction: data drift is a change in P(X) — the input feature distribution. Concept drift is a change in P(Y|X) — the conditional relationship between features and labels. Data drift does not always degrade model performance (if the new distribution is still within the learned decision boundary). Concept drift always degrades performance because the model's learned mapping from X to Y is now incorrect, regardless of whether X looks the same.
import numpy as np
import pandas as pd
# Scenario: credit scoring model
# Feature: income. Label: 1 = defaults, 0 = repays
# Training period: incomes 30-50k correlate with defaults
np.random.seed(42)
train_income = np.random.normal(40000, 8000, 1000)
train_default = (train_income < 35000).astype(int) # low income -> default
# Concept drift: inflation shifts threshold; now defaults start at 50k
prod_income = np.random.normal(40000, 8000, 500) # SAME distribution (no data drift)
prod_default = (prod_income < 50000).astype(int) # new relationship
print('Train default rate:', train_default.mean().round(3))
print('Prod default rate:', prod_default.mean().round(3))
print('Feature distribution same (no data drift).',
'But Y|X relationship has changed (concept drift).')Types of Concept Drift
Concept drift comes in four varieties. Sudden drift: the relationship changes abruptly (e.g., COVID-19 changing consumer spending patterns overnight). Gradual drift: the old concept slowly fades and a new concept emerges (e.g., shifting definition of spam over months). Incremental drift: a continuous, slow drift with no clear breakpoint. Recurring drift: seasonal patterns — the relationship at Christmas differs from July, but July eventually returns.
import numpy as np
import matplotlib.pyplot as plt
t = np.linspace(0, 100, 1000)
# Sudden drift: step function at t=50
sudden = np.where(t < 50, 0.8, 0.3) # accuracy drops at t=50
# Gradual drift: linear transition
gradual = np.where(t < 40, 0.8,
np.where(t > 60, 0.4,
0.8 - (t - 40) * 0.02))
# Recurring drift: seasonal pattern
recurring = 0.6 + 0.2 * np.sin(2 * np.pi * t / 20)
for name, series in [('Sudden', sudden), ('Gradual', gradual), ('Recurring', recurring)]:
print(f'{name} drift -- min acc: {series.min():.2f}, max: {series.max():.2f}')Detecting Concept Drift via Performance Monitoring
The most reliable signal for concept drift is declining model performance on labelled production data. Track your chosen metric (accuracy, F1, AUC) on a rolling window of recent predictions where ground-truth labels have become available. A downward trend in performance that is not explained by data drift (the features look the same) is strong evidence of concept drift. This requires a feedback loop: collecting true labels for production predictions, which can take days to months depending on the task.
import numpy as np
import pandas as pd
np.random.seed(42)
# Simulate weekly model accuracy tracking
weekly_accuracy = [
0.92, 0.91, 0.90, 0.89, 0.88, # slight decline
0.86, 0.83, 0.79, 0.74, 0.68, # accelerating decline = concept drift
0.65, 0.61
]
df = pd.DataFrame({'week': range(1, 13), 'accuracy': weekly_accuracy})
df['rolling_mean'] = df['accuracy'].rolling(3).mean()
df['alert'] = df['accuracy'] < 0.75 # threshold
print(df.to_string(index=False))
print('Weeks with alerts:', df[df['alert']]['week'].tolist())Detecting Concept Drift Without Labels
Waiting for ground-truth labels is slow. When labels arrive with high latency, use prediction distribution monitoring as an early warning signal. If P(X) is stable (no data drift) but the model's predicted label distribution shifts significantly, this suggests the underlying concept has changed. For example, if a binary classifier's positive-prediction rate drops from 30% to 5% without any change in feature distributions, the model is likely seeing a different relationship between features and the positive class.
import numpy as np
# Monitoring prediction distributions as a proxy for concept drift
# Period 1 (training distribution): ~30% positive predictions
period1_probs = np.random.beta(2, 5, 1000) # right-skewed, ~30% above 0.5
period1_pos_rate = (period1_probs > 0.5).mean()
# Period 2 (concept drifted): only ~5% positive predictions (model confused)
period2_probs = np.random.beta(1, 15, 500) # very right-skewed
period2_pos_rate = (period2_probs > 0.5).mean()
print(f'Period 1 positive rate: {period1_pos_rate:.2%}')
print(f'Period 2 positive rate: {period2_pos_rate:.2%}')
print(f'Rate drop: {(period1_pos_rate - period2_pos_rate):.2%}')
print('Possible concept drift detected!')Windowed Retraining: Adapting to Concept Drift
The most common strategy for handling concept drift is windowed retraining: periodically retrain the model using only the most recent N examples, discarding old data that reflects outdated patterns. A sliding window keeps the most recent N examples; an expanding window uses all data with recency weighting. The optimal window size depends on how fast the concept evolves — fast-changing domains (financial markets) need smaller windows than slow-changing ones (weather forecasting).
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
np.random.seed(42)
def simulate_concept_drift(n, t):
X = np.random.normal(0, 1, (n, 2))
# Decision boundary rotates over time
y = ((X[:, 0] * np.cos(t) - X[:, 1] * np.sin(t)) > 0).astype(int)
return X, y
# Full retrain vs sliding window retrain
for window_size in [100, 500, None]: # None = all data
all_X, all_y = [], []
accs = []
for t in np.linspace(0, np.pi, 10):
X, y = simulate_concept_drift(200, t)
all_X.append(X)
all_y.append(y)
if window_size:
train_X = np.vstack(all_X[-window_size//200:])
train_y = np.hstack(all_y[-window_size//200:])
else:
train_X = np.vstack(all_X)
train_y = np.hstack(all_y)
clf = LogisticRegression().fit(train_X, train_y)
accs.append(clf.score(X, y))
print(f'Window={window_size}: avg accuracy={np.mean(accs):.3f}')Sample Weighting: Downweighting Old Data
Instead of a hard window cutoff, assign exponentially decaying weights to training examples so recent examples contribute more to the model. Examples from last week get weight 1.0; examples from last month get weight 0.5; examples from six months ago get weight 0.1. Many scikit-learn estimators accept a sample_weight parameter in their fit method, making this easy to implement without discarding any data.
import numpy as np
from sklearn.ensemble import GradientBoostingClassifier
np.random.seed(42)
# Simulate 1000 training examples collected over 10 weeks
X = np.random.normal(0, 1, (1000, 5))
y = (X[:, 0] + np.random.normal(0, 0.5, 1000) > 0).astype(int)
weeks_ago = np.linspace(10, 0, 1000) # example 0 is oldest, 999 is newest
# Exponential decay: half-life of 3 weeks
half_life = 3.0
sample_weights = np.exp(-weeks_ago * np.log(2) / half_life)
sample_weights /= sample_weights.sum()
clf = GradientBoostingClassifier(n_estimators=100, random_state=42)
clf.fit(X, y, sample_weight=sample_weights * len(y)) # scale up
print('Model trained with recency-weighted samples.')
print('Weight ratio new/old:', round(sample_weights[-1] / sample_weights[0], 2))CUSUM and ADWIN Drift Detectors
Dedicated concept drift detection algorithms provide more principled alerts. CUSUM (Cumulative Sum) detects changes in the mean of a sequence by accumulating deviations from a reference value. ADWIN (Adaptive Windowing) maintains a variable-length window and shrinks it when a statistical test detects a distribution change within the window. The river library (formerly scikit-multiflow) implements both for streaming data.
# Using the river library for streaming drift detection
# pip install river
# from river.drift import ADWIN
# adwin = ADWIN(delta=0.002) # delta controls sensitivity
# error_sequence = [0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1] # errors over time
#
# for i, error in enumerate(error_sequence):
# adwin.update(error)
# if adwin.drift_detected:
# print(f'ADWIN detected drift at step {i}')
# adwin = ADWIN(delta=0.002) # reset detector
# Manual CUSUM example:
def cusum(errors, threshold=5, drift=0.01):
cum_sum, alerts = 0, []
for i, e in enumerate(errors):
cum_sum = max(0, cum_sum + e - drift)
if cum_sum > threshold:
alerts.append(i)
cum_sum = 0
return alerts
errors = [0]*10 + [1]*15 # errors spike after step 10
print('CUSUM drift alerts at steps:', cusum(errors))Concept Drift vs Seasonality
Seasonality is predictable periodic variation (Christmas shopping spikes, summer travel patterns) while concept drift is an unpredictable one-directional shift. A good monitoring system distinguishes them by checking historical seasonality patterns. If the current distribution matches last year's same-period distribution, it is likely seasonality rather than drift. Include time-of-year features in your model to handle predictable seasonality rather than triggering unnecessary retraining.
import numpy as np
from scipy.stats import ks_2samp
np.random.seed(42)
# Training: July 2023 data
july_2023 = np.random.normal(100, 20, 1000) # low activity (summer)
# Monitoring: July 2024 (seasonal -- same month last year)
july_2024 = np.random.normal(102, 21, 500) # similar to last July
# Monitoring: December 2024 (seasonal spike -- same as Dec 2023)
dec_2024 = np.random.normal(180, 30, 500) # high activity (Christmas)
stat_july, p_july = ks_2samp(july_2023, july_2024)
stat_dec, p_dec = ks_2samp(july_2023, dec_2024)
print(f'July 2024 vs July 2023: KS={stat_july:.3f} p={p_july:.3f} -> {"drift" if p_july < 0.05 else "seasonal OK"}')
print(f'Dec 2024 vs July 2023: KS={stat_dec:.3f} p={p_dec:.3f} -> Expected seasonal shift')Building a Concept Drift Response Plan
Respond to concept drift with a tiered plan. Tier 1 (early warning): alert on prediction distribution shifts without waiting for labels. Tier 2 (confirmation): when labels arrive, confirm performance degradation exceeds 5% relative. Tier 3 (action): trigger automatic windowed retraining using the drift detection timestamp as the window cutoff. Tier 4 (escalation): if retraining does not recover performance, escalate to a data scientist for feature engineering review.
def concept_drift_response(current_accuracy, baseline_accuracy,
data_drift_detected, prediction_rate_shift):
relative_drop = (baseline_accuracy - current_accuracy) / baseline_accuracy
if relative_drop > 0.10:
return ('TIER 4 ESCALATION: >10% accuracy drop. '
'Manual feature engineering review required.')
elif relative_drop > 0.05:
return ('TIER 3 ACTION: 5-10% accuracy drop. '
'Trigger windowed retraining immediately.')
elif data_drift_detected or prediction_rate_shift > 0.15:
return ('TIER 2 MONITOR: Early warning signals present. '
'Increase monitoring to daily. Queue retraining.')
else:
return 'TIER 1 OK: No significant concept drift detected.'
print(concept_drift_response(0.84, 0.92, False, 0.05)) # TIER 3
print(concept_drift_response(0.91, 0.92, True, 0.18)) # TIER 2
print(concept_drift_response(0.91, 0.92, False, 0.03)) # TIER 1Ensemble Models and Concept Drift Resilience
Ensemble methods like random forests and gradient boosting can mask concept drift for longer than single models because their diversity makes them less sensitive to any one feature distribution shift. However, this resilience is a double-edged sword: the model degrades more slowly and more silently. Monitor the individual tree disagreement rate in a forest — when trees increasingly disagree on predictions, it is an early signal that the concept is drifting and the ensemble is struggling to reach consensus.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
# Measure inter-tree disagreement on current data as drift signal
def inter_tree_disagreement(forest, X):
predictions = np.array([tree.predict(X) for tree in forest.estimators_])
# Fraction of pairs of trees that disagree per sample, averaged over samples
n_trees = len(forest.estimators_)
disagreement_per_sample = []
for i in range(X.shape[0]):
preds = predictions[:, i]
disagree = (preds != preds[0]).sum() / n_trees
disagreement_per_sample.append(disagree)
return np.mean(disagreement_per_sample)
# Higher disagreement = more concept drift
# print(inter_tree_disagreement(model, X_current))Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: concept drift is a change in P(Y|X) — the mapping from features to labels — which can occur even when feature distributions appear stable, declining model performance on labelled production data is the definitive signal of concept drift, and windowed retraining and sample weighting are the primary adaptation strategies to keep models current with evolving concepts. Next up we look at monitoring prediction distributions and confidence scores as an early warning system when ground-truth labels are not yet available.
Aprenda Python com um tutor de IA — grátis
Escreva e execute código real no seu navegador, obtenha ajuda instantânea de um tutor de IA 24/7 e continue de onde parou na web ou no app.
- Cursos
- 30
- Aulas
- 120
Perguntas Frequentes
A aula “Deriva de conceito: quando a relação entre X e Y muda” é grátis?
Sim — o texto completo de “Deriva de conceito: quando a relação entre X e Y muda” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de Machine Learning Academy, atualize para CoddyKit PRO. O curso de Machine Learning Academy inclui 4 aulas no total.
O que vou aprender em “Deriva de conceito: quando a relação entre X e Y muda”?
Os alunos distinguirão a deriva dos dados da deriva de conceito usando um exemplo de série temporal e entenderão por que a deriva dos dados nem sempre implica degradação do desempenho do modelo. Você pratica Machine Learning Academy com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.
Preciso ter experiência prévia para começar Machine Learning Academy?
Nenhuma experiência prévia é necessária. Machine Learning Academy no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 2 de 4.
Quanto tempo leva a aula “Deriva de conceito: quando a relação entre X e Y muda”?
A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.
Posso escrever e executar código nesta aula de Machine Learning Academy?
Sim. Cada aula de Machine Learning Academy inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.
Todas as aulas deste curso
- Deriva dos dados: mudanças na distribuição das características ao longo do tempo
- Deriva de conceito: quando a relação entre X e Y muda
- Monitoramento das distribuições de previsões e das pontuações de confiança
- Criando um pipeline de alertas de deriva com Evidently AI