اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس
سيحسب المتعلمون تكرارات الفئات، ويكشفون فخ المصنّف الوهمي في مجموعة بيانات غير متوازنة، ويتأكدون من أن الدقة مقياس مضلل في هذه الحالة.
اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس درس مجاني في Machine Learning Academy على CoddyKit. هذا هو الدرس 1 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في Machine Learning Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.
بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.
What Is Class Imbalance?
Class imbalance occurs when one class dominates the dataset. In fraud detection, fraudulent transactions may be 0.1% of all records; in medical diagnosis, a rare disease may affect 1 in 1000 patients. A model that always predicts the majority class achieves 99.9% accuracy while completely ignoring the rare, critical class — making accuracy a dangerously misleading metric in these scenarios.
Measuring Class Distribution
Before training any model, inspect the class distribution with pd.Series(y).value_counts() or np.bincount(y). A useful summary statistic is the imbalance ratio — the ratio of majority to minority class count. Ratios above 10:1 are considered imbalanced; above 100:1 are severely imbalanced and require specialised techniques.
import numpy as np
import pandas as pd
# Simulate imbalanced binary classification dataset
np.random.seed(0)
y = np.array([0] * 950 + [1] * 50) # 95% negative, 5% positive
counts = pd.Series(y).value_counts()
print('Class counts:\n', counts)
print()
print('Class proportions:\n', counts / len(y))
print()
print('Imbalance ratio:', counts[0] / counts[1])The Accuracy Trap with Imbalanced Data
On a dataset with 95% negative examples, a model that always predicts the majority class (negative) achieves 95% accuracy without learning anything useful. This is the accuracy trap. The model has zero ability to detect positive cases (the class you actually care about), yet its accuracy looks impressive. This is why imbalanced datasets demand metrics beyond accuracy.
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, classification_report
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05],
random_state=42, n_features=10)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
# Always predict majority class
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
y_pred = dummy.predict(X_test)
print('Accuracy:', accuracy_score(y_test, y_pred).round(4))
print()
print(classification_report(y_test, y_pred))Why the Dummy Classifier Baseline Matters
The DummyClassifier with strategy='most_frequent' is the minimum acceptable baseline. Any real model must beat it meaningfully — not just on accuracy but on the metrics that matter (precision, recall, F1, or AUC-ROC). If your real model is only marginally better than the dummy, it has likely learned to ignore the minority class too.
from sklearn.linear_model import LogisticRegression
from sklearn.dummy import DummyClassifier
from sklearn.metrics import classification_report
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)
lr = LogisticRegression().fit(X_train_s, y_train)
print('--- Logistic Regression ---')
print(classification_report(y_test, lr.predict(X_test_s)))Precision and Recall on Imbalanced Data
For imbalanced problems, the two most informative metrics are: Precision = TP / (TP + FP) — of all predicted positives, how many were actually positive? Recall = TP / (TP + FN) — of all actual positives, how many did the model catch? In fraud detection, recall is paramount (don't miss fraud); in spam filtering, precision matters (don't misclassify legitimate emails).
from sklearn.metrics import precision_score, recall_score, f1_score
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)
lr = LogisticRegression().fit(X_train_s, y_train)
y_pred = lr.predict(X_test_s)
print(f'Precision: {precision_score(y_test, y_pred):.4f}')
print(f'Recall: {recall_score(y_test, y_pred):.4f}')
print(f'F1-Score: {f1_score(y_test, y_pred):.4f}')ROC-AUC: Threshold-Agnostic Evaluation
ROC-AUC evaluates the model at all possible decision thresholds and measures the area under the resulting curve. An AUC of 0.5 is random; 1.0 is perfect; 0.9+ is excellent. Unlike accuracy, AUC is not affected by class imbalance because it evaluates the model's ranking ability rather than a fixed threshold prediction. AUC is the recommended primary metric for most imbalanced binary classification tasks.
from sklearn.metrics import roc_auc_score
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
sc = StandardScaler()
X_train_s = sc.fit_transform(X_train)
X_test_s = sc.transform(X_test)
lr = LogisticRegression().fit(X_train_s, y_train)
proba = lr.predict_proba(X_test_s)[:, 1]
print('ROC-AUC:', roc_auc_score(y_test, proba).round(4))Visualising Class Imbalance
A simple bar chart of class frequencies immediately communicates imbalance to stakeholders. Include it in your exploratory data analysis notebook as standard practice. If you are working with a Pandas DataFrame, also check for hidden imbalance caused by data collection bias rather than actual rarity in the real world.
import matplotlib.pyplot as plt
import numpy as np
y = np.array([0] * 950 + [1] * 50)
classes, counts = np.unique(y, return_counts=True)
plt.bar(['Negative (0)', 'Positive (1)'], counts, color=['#2196F3', '#F44336'])
plt.ylabel('Count')
plt.title('Class Distribution (95:5 imbalance)')
for i, c in enumerate(counts):
plt.text(i, c + 5, str(c), ha='center', fontweight='bold')
plt.show()Stratified Splitting to Preserve Ratios
When splitting an imbalanced dataset, use stratify=y in train_test_split. Without stratification, random chance might put all minority-class samples in one split, making training or evaluation meaningless. Stratified splitting ensures both train and test sets have the same class ratio as the original dataset.
from sklearn.model_selection import train_test_split
import numpy as np
y = np.array([0] * 950 + [1] * 50)
X = np.random.randn(1000, 5)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42
)
print('Train class ratio:', (y_train == 1).mean().round(4))
print('Test class ratio:', (y_test == 1).mean().round(4))
print('Expected ratio:', (y == 1).mean().round(4))Stratified K-Fold for Cross-Validation
Similarly, use StratifiedKFold (or simply pass cv=5 to cross_val_score — scikit-learn uses StratifiedKFold automatically for classifiers) to ensure each fold maintains the original class ratio. A regular KFold with small minority classes can create folds with zero positive examples, causing errors or misleading metrics.
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import numpy as np
y = np.array([0] * 950 + [1] * 50)
X = np.random.randn(1000, 5)
pipe = Pipeline([('sc', StandardScaler()), ('lr', LogisticRegression())])
skf = StratifiedKFold(n_splits=5)
scores = cross_val_score(pipe, X, y, cv=skf, scoring='roc_auc')
print(f'Stratified CV AUC: {np.mean(scores):.4f} +/- {np.std(scores):.4f}')The Precision-Recall Curve
For highly imbalanced data, the precision-recall (PR) curve is often more informative than the ROC curve. The PR curve shows precision on the y-axis and recall on the x-axis at all thresholds. The area under the PR curve (AP score) ranges from 0 to 1 and is sensitive to the minority-class performance in a way that ROC-AUC can mask when the majority class is overwhelming.
from sklearn.metrics import precision_recall_curve, average_precision_score
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
X, y = make_classification(n_samples=1000, weights=[0.95, 0.05], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=42)
sc = StandardScaler()
X_train_s = sc.fit_transform(X_train)
X_test_s = sc.transform(X_test)
lr = LogisticRegression().fit(X_train_s, y_train)
proba = lr.predict_proba(X_test_s)[:, 1]
precision, recall, _ = precision_recall_curve(y_test, proba)
ap = average_precision_score(y_test, proba)
plt.plot(recall, precision, label=f'AP={ap:.3f}')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.legend()
plt.show()Documenting Imbalance Before Modelling
Best practice: create an imbalance report at the start of every classification project. Record class counts, ratios, and the baseline dummy accuracy. This sets expectations and forces the team to agree on the right success metric before any model is trained. A fraud detection model with 95% accuracy is usually worthless — your stakeholders need to know this upfront.
import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, roc_auc_score
y = np.array([0] * 950 + [1] * 50)
X = np.random.randn(1000, 5)
dummy = DummyClassifier(strategy='most_frequent').fit(X, y)
dummy_acc = accuracy_score(y, dummy.predict(X))
dummy_auc = roc_auc_score(y, dummy.predict_proba(X)[:, 1])
print('=== Imbalance Report ===')
print(f'Class 0: {(y==0).sum()} ({(y==0).mean():.1%})')
print(f'Class 1: {(y==1).sum()} ({(y==1).mean():.1%})')
print(f'Imbalance ratio: {(y==0).sum()/(y==1).sum():.0f}:1')
print(f'Dummy accuracy: {dummy_acc:.4f}')
print(f'Dummy AUC: {dummy_auc:.4f}')
print('Recommended metric: ROC-AUC or Precision-Recall AUC')Quick Check
Test your understanding of class imbalance and baseline pitfalls from this lesson.
Lesson Recap
In this lesson you learned: class imbalance makes accuracy a misleading metric — a model predicting only the majority class can achieve very high accuracy, always compare against a DummyClassifier baseline to ensure your model is learning something beyond the trivial, and use ROC-AUC or precision-recall AUC as primary metrics for imbalanced datasets. Next up we apply SMOTE and random oversampling to upsample the minority class.
الأسئلة الشائعة
هل درس «اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس» مجاني؟
نعم — نص درس «اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة Machine Learning Academy، انتقل إلى CoddyKit PRO. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.
ماذا ستتعلم في «اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس»؟
سيحسب المتعلمون تكرارات الفئات، ويكشفون فخ المصنّف الوهمي في مجموعة بيانات غير متوازنة، ويتأكدون من أن الدقة مقياس مضلل في هذه الحالة. تتمرن على Machine Learning Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.
هل أحتاج إلى خبرة سابقة لأبدأ Machine Learning Academy؟
لا تُشترط خبرة سابقة. Machine Learning Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 1 من أصل 4.
كم من الوقت يستغرق درس «اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس»؟
معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.
هل يمكنني كتابة وتشغيل أكواد في درس Machine Learning Academy هذا؟
نعم. كل درس في Machine Learning Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.
جميع الدروس في هذه الدورة
- اكتشاف عدم التوازن: توزيع الفئات ومزالق خط الأساس
- إعادة أخذ العينات العشوائية الزائدة وSMOTE
- إعادة أخذ العينات العشوائية الناقصة ومراكز العناقيد
- أوزان الفئات وتحريك العتبة