0Pricing
Machine Learning Academy · درس

اختيار السمات العشوائي: حيلة Random Forest

اضبط max_features في RandomForestClassifier، ولاحظ كيف يفصل أخذ عينات فرعية من السمات الارتباط بين الأشجار، وشاهد ارتفاع دقة الاختبار

اختيار السمات العشوائي: حيلة Random Forest درس مجاني في Machine Learning Academy على CoddyKit. هذا هو الدرس 2 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في Machine Learning Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.

بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.

From Bagging to Random Forests

A plain bagging ensemble trains each tree on a different bootstrap sample, but all trees can still use every feature when choosing a split. This means the most predictive feature will dominate every tree, making them highly correlated. When correlated models are averaged, variance reduction is limited. Random Forests add one extra trick: at every split, only a random subset of features is considered. This decorrelates the trees and dramatically improves the ensemble's generalisation.

The max_features Parameter

In RandomForestClassifier, the max_features parameter controls how many features are candidates at each split. Common choices are 'sqrt' (square root of the total number of features, the default for classification), 'log2', or an integer/float. For regression (RandomForestRegressor), the default is 1.0 (all features), and 'sqrt' or 0.33 are popular alternatives. Smaller values create more diverse trees at the cost of each tree being slightly weaker individually.

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import cross_val_score
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
for mf in ['sqrt', 'log2', 0.5, 1.0]:
    rf = RandomForestClassifier(n_estimators=100, max_features=mf, random_state=42)
    score = cross_val_score(rf, X, y, cv=5).mean()
    print(f'max_features={str(mf):6s}: CV accuracy={score:.4f}')

Why Feature Sub-Sampling Decorrelates Trees

Consider a dataset with one dominant feature that predicts the label far better than any other. Without feature sub-sampling, every tree in the ensemble will split on that feature at the root, producing nearly identical trees. Averaging identical predictions yields the same prediction — no variance reduction. When only sqrt(p) features are candidates at each node, that dominant feature is absent in many splits, forcing trees to discover alternative predictive patterns and become genuinely diverse.

Training a RandomForestClassifier

Using RandomForestClassifier from scikit-learn is straightforward. You specify the number of trees with n_estimators, control tree complexity with max_depth, and enable OOB evaluation with oob_score=True. The model trains all trees in parallel when n_jobs=-1. After fitting, feature_importances_ gives you a ranked measure of which input features drove the predictions across all trees.

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

rf = RandomForestClassifier(
    n_estimators=200,
    max_features='sqrt',
    oob_score=True,
    n_jobs=-1,
    random_state=42
)
rf.fit(X_train, y_train)
print('OOB score:', rf.oob_score_)
print('Test accuracy:', rf.score(X_test, y_test))

Feature Importances from Random Forest

Random Forests compute feature importance as the average decrease in impurity (Gini or entropy) caused by splits on that feature, weighted by the number of samples that pass through each node, averaged across all trees. This gives a fast, global ranking of predictive power. The importances sum to 1 and are accessible via rf.feature_importances_. However, they can be biased toward high-cardinality features, so treat them as a useful heuristic rather than an absolute ground truth.

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
X, y = data.data, data.target
feature_names = data.feature_names

rf = RandomForestClassifier(n_estimators=200, random_state=42)
rf.fit(X, y)

importances = pd.Series(rf.feature_importances_, index=feature_names)
print(importances.sort_values(ascending=False).head(5))

Comparing Random Forest to a Single Tree

A single decision tree trained on the full dataset is highly sensitive to noise in the training data — changing a few examples can drastically alter the tree structure. A random forest, by averaging hundreds of such sensitive trees, creates a prediction surface that is much smoother and more robust. This comparison is one of the clearest demonstrations of how ensemble diversity translates into generalisation improvement.

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import cross_val_score
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
single = DecisionTreeClassifier(random_state=42)
forest = RandomForestClassifier(n_estimators=200, random_state=42)

print('Single tree CV:', np.round(cross_val_score(single, X, y, cv=5), 4))
print('Random forest CV:', np.round(cross_val_score(forest, X, y, cv=5), 4))

Controlling Tree Depth in a Forest

Each tree in a Random Forest is typically grown deep (low bias, high variance). Bagging then corrects the variance. However, for very large datasets you may limit max_depth or min_samples_leaf to reduce memory and training time. Setting max_depth=None (the default) lets trees grow until all leaves are pure. A shallow forest behaves more like a boosting ensemble: low-variance base learners that struggle to capture complex boundaries. The sweet spot depends on your dataset's noise level.

Random Forest for Regression

RandomForestRegressor follows the same algorithm but averages the predicted numeric values from each tree rather than voting on a class label. The default max_features for regression in modern scikit-learn is 1.0, but using 'sqrt' or a fraction often helps. Regression forests work well on tabular data with non-linear interactions and are frequently used for house price prediction, energy consumption forecasting, and other continuous-target problems.

from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import cross_val_score
import numpy as np

X, y = fetch_california_housing(return_X_y=True)
rfr = RandomForestRegressor(n_estimators=100, max_features='sqrt', n_jobs=-1, random_state=42)
rmse = np.sqrt(-cross_val_score(rfr, X, y, scoring='neg_mean_squared_error', cv=3).mean())
print(f'Random Forest Regression RMSE: {rmse:.4f}')

Variance of Feature Importances

Because each tree sees a different bootstrap sample and different feature subsets, the feature importances computed from different runs or different forests can vary. For more reliable importance estimates, use permutation importance (sklearn.inspection.permutation_importance), which measures how much model accuracy drops when a feature's values are randomly shuffled. Permutation importance works for any black-box model and does not suffer from the high-cardinality bias of impurity-based importances.

from sklearn.inspection import permutation_importance
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
import pandas as pd

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
rf = RandomForestClassifier(n_estimators=100, random_state=42).fit(X_train, y_train)
result = permutation_importance(rf, X_test, y_test, n_repeats=10, random_state=42)
print(pd.Series(result.importances_mean, index=load_breast_cancer().feature_names).sort_values(ascending=False).head(5))

Hyperparameter Tuning for Random Forests

The most important hyperparameters in a Random Forest are n_estimators (more is usually better, up to a plateau), max_features (controls decorrelation), max_depth and min_samples_leaf (control bias-variance trade-off per tree). A grid search over max_features and min_samples_leaf is often sufficient because n_estimators can be set high and max_depth=None is a good default. Use OOB score for fast approximate tuning before committing to full cross-validation.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
param_grid = {'max_features': ['sqrt', 'log2', 0.5], 'min_samples_leaf': [1, 3, 5]}
grid = GridSearchCV(RandomForestClassifier(n_estimators=100, random_state=42), param_grid, cv=5)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best CV score:', round(grid.best_score_, 4))

Limitations of Random Forests

Random Forests are powerful but have notable limitations. They are memory-intensive because every tree must be stored. Inference is slow on very large forests since each prediction requires traversing all trees. They are also not the best for structured tabular data with many irrelevant features, where gradient boosting often wins. Finally, Random Forests lack a natural mechanism to capture sequential or spatial structure — for images or text, deep learning methods outperform them decisively.

Quick Check

Test your understanding of Random Forest feature selection concepts from this lesson.

Lesson Recap

In this lesson you learned: Random Forests add feature sub-sampling to bagging to decorrelate trees, max_features controls the number of candidate features at each split, and feature importances reveal which inputs drive predictions across the forest. Next up we explore out-of-bag error as a free built-in validation mechanism.

الأسئلة الشائعة

هل درس «اختيار السمات العشوائي: حيلة Random Forest» مجاني؟

نعم — نص درس «اختيار السمات العشوائي: حيلة Random Forest» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة Machine Learning Academy، انتقل إلى CoddyKit PRO. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.

ماذا ستتعلم في «اختيار السمات العشوائي: حيلة Random Forest»؟

اضبط max_features في RandomForestClassifier، ولاحظ كيف يفصل أخذ عينات فرعية من السمات الارتباط بين الأشجار، وشاهد ارتفاع دقة الاختبار تتمرن على Machine Learning Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.

هل أحتاج إلى خبرة سابقة لأبدأ Machine Learning Academy؟

لا تُشترط خبرة سابقة. Machine Learning Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 2 من أصل 4.

كم من الوقت يستغرق درس «اختيار السمات العشوائي: حيلة Random Forest»؟

معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.

هل يمكنني كتابة وتشغيل أكواد في درس Machine Learning Academy هذا؟

نعم. كل درس في Machine Learning Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.

جميع الدروس في هذه الدورة

  1. شرح التجميع بالت bootstrap ‏(Bagging)
  2. اختيار السمات العشوائي: حيلة Random Forest
  3. خطأ Out-of-Bag: تحقق مجاني داخل الغابة
  4. تجميع التصويت: التصويت الصارم في مقابل التصويت المرن
← العودة إلى Machine Learning Academy