Class Weights and Threshold Moving
Learners will set class_weight='balanced' in sklearn classifiers and then shift the decision threshold on probability outputs to further optimise recall for rare events.
Two Alternatives to Resampling
Resampling (SMOTE, undersampling) physically changes the training data. Two alternative approaches work directly with the original data: class weighting penalises misclassifying minority samples more heavily during training, and threshold moving adjusts the decision boundary after training. Both are simpler, faster, and avoid the information loss or synthetic-noise risks of resampling.
Class Weights: Penalise Minority Errors More
Most sklearn classifiers accept a class_weight parameter. Setting class_weight='balanced' automatically computes weights inversely proportional to class frequencies: weight[c] = n_samples / (n_classes * count[c]). Misclassifying a rare positive sample is then penalised much more than misclassifying a common negative sample, pushing the model to learn the minority class better.
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=42)
sc = StandardScaler()
X_tr = sc.fit_transform(X_train)
X_te = sc.transform(X_test)
# With balanced class weights
lr = LogisticRegression(class_weight='balanced').fit(X_tr, y_train)
print('--- class_weight=balanced ---')
print(classification_report(y_test, lr.predict(X_te)))All lessons in this course
- Detecting Imbalance: Class Distribution and Baseline Pitfalls
- Random Oversampling and SMOTE
- Random Undersampling and Cluster Centroids
- Class Weights and Threshold Moving