البحث الشبكي مقابل البحث العشوائي
سيضبط المتعلمون GridSearchCV وRandomizedSearchCV على مساحة المعاملات الفائقة نفسها، ويقارنون بين مدى تغطيتهما وتكلفة الحساب، ويختارون الأسرع للمساحات الكبيرة.
البحث الشبكي مقابل البحث العشوائي درس مجاني في Machine Learning Academy على CoddyKit. هذا هو الدرس 3 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في Machine Learning Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.
بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.
The Hyperparameter Search Problem
Most machine learning models have multiple hyperparameters that cannot be learned from data and must be set by the practitioner. Finding the optimal combination manually is impractical — the number of combinations grows exponentially with the number of hyperparameters. Two systematic approaches dominate: Grid Search, which evaluates every combination on a predefined grid, and Random Search, which samples combinations randomly from specified distributions. Understanding when to use each is a crucial practical skill.
Grid Search: Exhaustive Evaluation
GridSearchCV evaluates every combination of the hyperparameter values you specify. For a grid with 4 values of C, 5 values of gamma, and 5 CV folds, it trains 4 × 5 × 5 = 100 models. This guarantees you find the best combination within your grid. However, the cost grows multiplicatively: adding a third hyperparameter with 4 values multiplies the search to 400 models. Grid search works well when you have 1-2 hyperparameters and a manageable grid size.
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC())])
param_grid = {'svc__C': [0.1, 1, 10], 'svc__gamma': [0.01, 0.1, 1]}
# 3 * 3 * 5 folds = 45 model fits
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best CV score:', round(grid.best_score_, 4))The Curse of Dimensionality in Grid Search
Grid search suffers from exponential scaling. A model with 5 hyperparameters, each with 5 candidate values, requires 5^5 = 3125 model fits (times the number of CV folds). With 5-fold CV, that is 15,625 training runs. Even if each takes 1 second, that is 4+ hours. Practitioners often restrict grid search to the most important 1-2 hyperparameters, fix others at sensible defaults, and use coarse grids first. This heuristic works but risks missing interactions between hyperparameters not searched jointly.
Random Search: Sampling Instead of Grid
RandomizedSearchCV samples a fixed number of hyperparameter combinations (controlled by n_iter) randomly from specified distributions rather than testing every grid point. Research by Bergstra and Bengio (2012) showed that for the same computational budget, random search finds better hyperparameters than grid search when only a few hyperparameters strongly influence model performance — because random search effectively explores more distinct values of the important parameters.
from sklearn.svm import SVC
from sklearn.model_selection import RandomizedSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
from scipy.stats import loguniform
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC())])
param_dist = {'svc__C': loguniform(0.01, 100), 'svc__gamma': loguniform(0.0001, 1)}
# 30 random combinations * 5 folds = 150 model fits (same as a 5*6 grid)
rnd = RandomizedSearchCV(pipe, param_dist, n_iter=30, cv=5, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print('Best params:', {k: round(v, 5) for k, v in rnd.best_params_.items()})
print('Best CV score:', round(rnd.best_score_, 4))Why Random Search Wins for Many Hyperparameters
Consider 9 hyperparameters, 3 of which matter and 6 of which are irrelevant. A grid with 3 values per parameter tests 3^9 = 19683 combinations but only tests 3 distinct values for each important parameter. With the same budget of 19,683 random samples, each important parameter is explored with 19,683 distinct values. Concentrating samples in the space of important parameters gives random search a decisive advantage. The probability that at least one random configuration is in the top 5% of the space with just 60 iterations is over 95%.
Distributions for Random Search
Choosing appropriate probability distributions for RandomizedSearchCV is important. Use scipy.stats.loguniform(a, b) for parameters that span orders of magnitude like learning_rate or C. Use scipy.stats.uniform(a, b-a) for parameters with a linear scale like subsample (0.5 to 1.0). Use scipy.stats.randint(low, high) for integer parameters like n_estimators or max_depth. Using list-valued parameters (e.g., [3, 5, 7, 9]) samples uniformly from those discrete options.
from scipy.stats import loguniform, uniform, randint
import numpy as np
# Example distributions for RandomForestClassifier + LogisticRegression pipeline
param_dist = {
'rf__n_estimators': randint(50, 500), # integer, uniform
'rf__max_depth': [3, 5, 7, None], # discrete list
'rf__min_samples_leaf': randint(1, 20), # integer, uniform
'rf__max_features': loguniform(0.1, 1.0) # continuous, log-scale
}
# Show 5 sample combinations
np.random.seed(42)
for _ in range(3):
sample = {k: v.rvs() if hasattr(v, 'rvs') else np.random.choice(v) for k, v in param_dist.items()}
print(sample)Comparing Grid and Random Search Side by Side
A direct comparison: grid search with 5 values for each of 3 parameters requires 125 fits; random search with 125 iterations covers the same budget but explores continuous distributions instead of 5 fixed points per parameter. In practice, for simple 2D searches (C and gamma for SVM), grid search is perfectly adequate. For complex models like gradient boosting with 6+ hyperparameters, random search with 50-100 iterations consistently outperforms a grid with equivalent computation.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV, cross_val_score
from sklearn.datasets import load_breast_cancer
from scipy.stats import randint
import time, numpy as np
X, y = load_breast_cancer(return_X_y=True)
# Grid search
start = time.time()
grid = GridSearchCV(RandomForestClassifier(random_state=42), {'n_estimators': [50,100,200], 'max_depth': [3,5,None]}, cv=3, n_jobs=-1)
grid.fit(X, y)
print(f'Grid search: {round(time.time()-start,1)}s, best={round(grid.best_score_,4)}')
# Random search
start = time.time()
rnd = RandomizedSearchCV(RandomForestClassifier(random_state=42), {'n_estimators': randint(10,300), 'max_depth': [3,5,7,None]}, n_iter=9, cv=3, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print(f'Random search: {round(time.time()-start,1)}s, best={round(rnd.best_score_,4)}')Halving Grid Search for Larger Spaces
scikit-learn 0.24+ introduced HalvingGridSearchCV and HalvingRandomSearchCV based on the successive halving algorithm: start with all candidates trained on a small data subset, eliminate the worst half, double the data, repeat until one winner remains. This finds good hyperparameters with far less computation than full grid or random search, making it practical for larger search spaces with expensive models.
from sklearn.experimental import enable_halving_search_cv # noqa
from sklearn.model_selection import HalvingRandomSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from scipy.stats import randint
X, y = load_breast_cancer(return_X_y=True)
halving = HalvingRandomSearchCV(
RandomForestClassifier(random_state=42),
{'n_estimators': randint(10, 500), 'max_depth': [3, 5, 7, None], 'min_samples_leaf': randint(1, 20)},
cv=3, factor=2, random_state=42, n_jobs=-1
)
halving.fit(X, y)
print('Best params:', halving.best_params_)
print('Best score:', round(halving.best_score_, 4))Bayesian Optimisation: The Smart Alternative
Both grid and random search are uninformed — they do not use results from previous evaluations to guide future choices. Bayesian optimisation builds a probabilistic model of the objective function (the CV score as a function of hyperparameters) and uses it to intelligently select the most promising combination to evaluate next. Libraries like Optuna, BayesSearchCV (scikit-optimize), and HyperOpt implement this and typically find better hyperparameters in far fewer evaluations than random search.
Practical Search Strategy Recommendations
A practical guideline: (1) start with random search for 30-100 iterations to identify the promising hyperparameter region; (2) if you need more precision, run a fine grid search around that region; (3) for expensive models (slow training), use Bayesian optimisation (Optuna) to minimise evaluations; (4) always use a Pipeline to prevent leakage; (5) set n_jobs=-1 for parallelism; (6) use refit=True (default) so the best model is refitted on the full training data after search. Never use the test set to pick hyperparameters.
Accessing All Results from GridSearchCV
After fitting, grid.cv_results_ is a dictionary containing the mean test score, standard deviation, fit times, and parameter values for every combination evaluated. Converting it to a Pandas DataFrame makes it easy to sort, filter, and visualise. This is useful for understanding the sensitivity of the score to each parameter — if all C values produce similar scores but gamma has a big effect, you know future tuning should focus on gamma.
import pandas as pd
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
grid = GridSearchCV(pipe, {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}, cv=5)
grid.fit(X, y)
df = pd.DataFrame(grid.cv_results_)[['param_svc__C', 'param_svc__gamma', 'mean_test_score', 'std_test_score']]
print(df.sort_values('mean_test_score', ascending=False).head(5).round(4))Quick Check
Test your understanding of Grid Search vs Random Search from this lesson.
Lesson Recap
In this lesson you learned: Grid Search evaluates every combination exhaustively but scales exponentially with hyperparameter count, Random Search samples continuously from distributions and wins when few hyperparameters matter, and Halving and Bayesian optimisation further reduce evaluation cost for expensive models. Next up we explore Nested Cross-Validation for simultaneously selecting and evaluating hyperparameters without bias.
الأسئلة الشائعة
هل درس «البحث الشبكي مقابل البحث العشوائي» مجاني؟
نعم — نص درس «البحث الشبكي مقابل البحث العشوائي» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة Machine Learning Academy، انتقل إلى CoddyKit PRO. تتضمن دورة Machine Learning Academy 4 دروس في المجموع.
ماذا ستتعلم في «البحث الشبكي مقابل البحث العشوائي»؟
سيضبط المتعلمون GridSearchCV وRandomizedSearchCV على مساحة المعاملات الفائقة نفسها، ويقارنون بين مدى تغطيتهما وتكلفة الحساب، ويختارون الأسرع للمساحات الكبيرة. تتمرن على Machine Learning Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.
هل أحتاج إلى خبرة سابقة لأبدأ Machine Learning Academy؟
لا تُشترط خبرة سابقة. Machine Learning Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 3 من أصل 4.
كم من الوقت يستغرق درس «البحث الشبكي مقابل البحث العشوائي»؟
معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.
هل يمكنني كتابة وتشغيل أكواد في درس Machine Learning Academy هذا؟
نعم. كل درس في Machine Learning Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.
جميع الدروس في هذه الدورة
- التحقق المتقاطع بنظام K-Fold: التقسيم دون تسرّب
- التحقق المتقاطع الطبقي والزمني
- البحث الشبكي مقابل البحث العشوائي
- التحقق المتقاطع المتداخل: الاختيار والتقييم في آن واحد