0Pricing
Machine Learning Academy · درس

النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي

أنشئ DummyClassifier باعتباره خطًا أساسيًا أدنى، وتأكد من أن كل نموذج حقيقي يتفوق عليه قبل اعتباره مفيدًا

النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي درس مجاني في Machine Learning Academy على CoddyKit. هذا هو الدرس 4 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في Machine Learning Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.

بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.

Why You Need a Baseline

When you report that your model achieves 92% accuracy, that number is meaningless without context. Is 92% impressive or disappointing? It depends entirely on what the simplest possible strategy would achieve on the same problem.

A baseline model is the floor that every real model must beat to be considered useful. Without a baseline, you might celebrate 92% accuracy on a dataset where a model that always predicts 'not fraud' would achieve 95% accuracy — which means your fancy ML model is actually worse than doing nothing.

The DummyClassifier: scikit-learn's Baseline Tool

Scikit-learn provides DummyClassifier as a minimal classifier that makes predictions using simple rules completely ignorant of the input features. It is the formal tool for establishing a baseline before any real modelling begins.

DummyClassifier is not a joke or placeholder — it is a rigorous sanity check. If your real model cannot beat the DummyClassifier, something is fundamentally wrong: your features may not be predictive, your pipeline has a bug, or the problem is harder than expected.

from sklearn.dummy import DummyClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Most frequent baseline: always predicts the majority class
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
baseline_acc = dummy.score(X_test, y_test)
print(f'Baseline accuracy (always predict majority class): {baseline_acc:.3f}')

DummyClassifier Strategies

DummyClassifier supports several baseline strategies:

  • most_frequent: always predicts the class that appears most often in training. Best for imbalanced datasets.
  • stratified: randomly predicts each class with the probability equal to its training frequency. Maintains class distribution.
  • uniform: randomly predicts each class with equal probability. Useful when all classes are equally represented.
  • constant: always predicts a specified constant class. Use to test what happens if you always predict the positive class.
from sklearn.dummy import DummyClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import f1_score
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

for strategy in ['most_frequent', 'stratified', 'uniform', 'prior']:
    dummy = DummyClassifier(strategy=strategy, random_state=42)
    dummy.fit(X_train, y_train)
    acc = dummy.score(X_test, y_test)
    print(f'strategy={strategy}: accuracy={acc:.3f}')

The Accuracy Paradox in Action

The accuracy paradox is most dramatic on imbalanced datasets. Consider credit card fraud where 99% of transactions are legitimate. The most_frequent DummyClassifier predicts 'not fraud' for every transaction and achieves 99% accuracy. A real ML model scoring 97% accuracy would look worse than this baseline by accuracy — even if it correctly identifies most of the actual fraud cases.

This is why the DummyClassifier baseline must be evaluated with the same metric you will use to evaluate your real model. For imbalanced problems, use F1, precision, or recall — not accuracy.

from sklearn.dummy import DummyClassifier
from sklearn.metrics import f1_score, accuracy_score
import numpy as np

# Simulate 99% negative class (not fraud)
np.random.seed(42)
n = 10000
y_true = np.array([0]*9900 + [1]*100)
X_fake = np.random.randn(n, 5)  # random features (not predictive)

# Baseline always predicts not-fraud
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_fake, y_true)
y_pred_dummy = dummy.predict(X_fake)

print(f'Dummy Accuracy: {accuracy_score(y_true, y_pred_dummy):.3f}')  # 0.990
print(f'Dummy F1-score: {f1_score(y_true, y_pred_dummy):.3f}')         # 0.000
print('Dummy catches 0 fraud cases despite 99% accuracy!')

DummyRegressor: Baseline for Regression

For regression problems, scikit-learn provides DummyRegressor with strategies:

  • mean: always predicts the training set mean. This is the standard baseline — R² measures how much better your model is than simply predicting the mean.
  • median: always predicts the training set median. More robust to outliers.
  • quantile: always predicts a specific quantile of the training target.
  • constant: always predicts a specified constant.

A regression model with R² below 0 is actually worse than always predicting the mean — a clear sign something went wrong in training.

from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split

X, y = fetch_california_housing(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

dummy_reg = DummyRegressor(strategy='mean')
dummy_reg.fit(X_train, y_train)
y_pred_d = dummy_reg.predict(X_test)

print(f'DummyRegressor MAE: {mean_absolute_error(y_test, y_pred_d):.3f}')
print(f'DummyRegressor R2:  {r2_score(y_test, y_pred_d):.3f}')  # exactly 0.0

Your Model vs the Baseline

Now compare your real model against the baseline using the same metric. The improvement over the baseline tells you how much value your ML model adds. This comparison should be the first result you report in any ML project.

If the improvement is small (e.g., real model achieves F1=0.65 vs baseline F1=0.60), ask whether the complexity and cost of the ML model is worth a 5% improvement. Sometimes a simple rule-based system is cheaper and sufficiently accurate.

from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.metrics import f1_score
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Baseline
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
baseline_f1 = f1_score(y_test, dummy.predict(X_test))

# Real model
pipeline = Pipeline([('scaler', StandardScaler()), ('clf', LogisticRegression(max_iter=1000))])
pipeline.fit(X_train, y_train)
real_f1 = f1_score(y_test, pipeline.predict(X_test))

print(f'Baseline F1: {baseline_f1:.3f}')
print(f'Real model F1: {real_f1:.3f}')
print(f'Improvement over baseline: {real_f1 - baseline_f1:.3f}')

Human-Level Performance as a Second Baseline

For many real-world problems, a human-level performance benchmark is more relevant than a dummy baseline. For example, a radiologist's error rate on detecting tumours, a human spam screener's accuracy, or an expert appraiser's house price estimation error.

Human performance provides a ceiling: if your model matches or exceeds human performance, you have solved the problem. If your model is far below human performance, there is room to grow. If your model is very close to human performance, improving it further may require extraordinary effort for diminishing returns.

Previous System as a Baseline

In industry, the most relevant baseline is usually the existing system your model is replacing. If the current production system uses hand-crafted rules, it is the target to beat. If a previous ML model has been deployed, its production metrics are your baseline.

When reporting results to stakeholders, always compare to the current system, not just a DummyClassifier. Business value comes from improvement over the status quo. A 2% improvement over the current system may be worth millions of dollars in reduced fraud losses, even if it seems small in absolute terms.

Baseline for Multi-Class Classification

For multi-class problems, the DummyClassifier baseline depends on class balance. With K balanced classes, the most_frequent strategy achieves 1/K accuracy. With imbalanced classes, it achieves the proportion of the most frequent class.

For metrics like macro-averaged F1 (which weights all classes equally), a random classifier will score much lower on rare classes. Always compare per-class metrics between your real model and the baseline to understand where the improvement comes from.

from sklearn.dummy import DummyClassifier
from sklearn.metrics import classification_report
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
y_pred = dummy.predict(X_test)

print('DummyClassifier (most_frequent) on Iris:')
print(classification_report(y_test, y_pred,
      target_names=['setosa', 'versicolor', 'virginica']))
print('Notice: only the majority class has non-zero precision/recall')

Zero-Rule Classifier: The Simplest Baseline

Even simpler than the DummyClassifier is the Zero-Rule (ZeroR) classifier — a common baseline in data science competitions. For classification, ZeroR always predicts the majority class. For regression, it always predicts the mean. ZeroR represents the absolute minimum that any model must beat to be useful.

If you cannot beat ZeroR, your features contain no information about the target variable. This diagnostic is fast and free to compute and should always be the first step in any new ML project before you spend time on complex models.

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, f1_score

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# ZeroR: manually compute
majority_class = np.bincount(y_train).argmax()
y_pred_zeror = np.full(len(y_test), majority_class)

print(f'Majority class in training: {majority_class}')
print(f'ZeroR Accuracy: {accuracy_score(y_test, y_pred_zeror):.3f}')
print(f'ZeroR F1 (pos): {f1_score(y_test, y_pred_zeror):.3f}')

Bagging Baselines in Your ML Projects

A complete baseline evaluation workflow for any ML project:

  1. Compute the DummyClassifier (most_frequent) or DummyRegressor (mean) baseline.
  2. If available, compute the current system baseline using its predictions on your test set.
  3. Research human-level performance from benchmarks or literature.
  4. Train your first simple ML model (logistic regression or a shallow decision tree) and compare all three.
  5. Only proceed to complex models if the simple model beats the baseline by a meaningful margin — complexity should be motivated by measurable improvement.
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)

models = {
    'DummyClassifier': DummyClassifier(strategy='most_frequent'),
    'LogisticRegression': Pipeline([('sc', StandardScaler()), ('clf', LogisticRegression(max_iter=1000))]),
    'DecisionTree(d=5)': DecisionTreeClassifier(max_depth=5),
    'RandomForest': RandomForestClassifier(random_state=42),
}

for name, model in models.items():
    score = cross_val_score(model, X, y, cv=5, scoring='f1').mean()
    print(f'{name}: F1={score:.3f}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: DummyClassifier establishes the floor of performance that every real model must exceed to provide value, accuracy baselines are misleading for imbalanced datasets — always use the same metric for baseline and real model comparison, and a complete baseline includes the trivial classifier, the current production system, and human-level performance when available. Next up we begin the Data Preprocessing Pipeline course, starting with systematic strategies for handling missing values using imputation techniques.

الأسئلة الشائعة

هل درس «النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي» مجاني؟

نعم — نص درس «النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة Machine Learning Academy، انتقل إلى CoddyKit PRO. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.

ماذا ستتعلم في «النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي»؟

أنشئ DummyClassifier باعتباره خطًا أساسيًا أدنى، وتأكد من أن كل نموذج حقيقي يتفوق عليه قبل اعتباره مفيدًا تتمرن على Machine Learning Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.

هل أحتاج إلى خبرة سابقة لأبدأ Machine Learning Academy؟

لا تُشترط خبرة سابقة. Machine Learning Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 4 من أصل 4.

كم من الوقت يستغرق درس «النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي»؟

معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.

هل يمكنني كتابة وتشغيل أكواد في درس Machine Learning Academy هذا؟

نعم. كل درس في Machine Learning Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.

جميع الدروس في هذه الدورة

  1. لماذا لا يمكنكم التقييم باستخدام بيانات التدريب
  2. train_test_split: النسب والبذور والتقسيم الطبقي
  3. مفاضلة التحيز والتباين: نقص الملاءمة في مقابل فرط الملاءمة
  4. النماذج الأساسية: تفوّق دائمًا على المصنّف الوهمي
← العودة إلى Machine Learning Academy