网格搜索与随机搜索
您将在相同的超参数空间上配置 GridSearchCV 和 RandomizedSearchCV，比较它们的覆盖范围与计算成本，并为大型空间选择更快的方法。
网格搜索与随机搜索 是 CoddyKit 上的免费 Machine Learning Academy 课时。 这是第 3 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
The Hyperparameter Search Problem
Most machine learning models have multiple hyperparameters that cannot be learned from data and must be set by the practitioner. Finding the optimal combination manually is impractical — the number of combinations grows exponentially with the number of hyperparameters. Two systematic approaches dominate: Grid Search, which evaluates every combination on a predefined grid, and Random Search, which samples combinations randomly from specified distributions. Understanding when to use each is a crucial practical skill.
Grid Search: Exhaustive Evaluation
GridSearchCV evaluates every combination of the hyperparameter values you specify. For a grid with 4 values of C, 5 values of gamma, and 5 CV folds, it trains 4 × 5 × 5 = 100 models. This guarantees you find the best combination within your grid. However, the cost grows multiplicatively: adding a third hyperparameter with 4 values multiplies the search to 400 models. Grid search works well when you have 1-2 hyperparameters and a manageable grid size.
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC())])
param_grid = {'svc__C': [0.1, 1, 10], 'svc__gamma': [0.01, 0.1, 1]}
# 3 * 3 * 5 folds = 45 model fits
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best CV score:', round(grid.best_score_, 4))The Curse of Dimensionality in Grid Search
Grid search suffers from exponential scaling. A model with 5 hyperparameters, each with 5 candidate values, requires 5^5 = 3125 model fits (times the number of CV folds). With 5-fold CV, that is 15,625 training runs. Even if each takes 1 second, that is 4+ hours. Practitioners often restrict grid search to the most important 1-2 hyperparameters, fix others at sensible defaults, and use coarse grids first. This heuristic works but risks missing interactions between hyperparameters not searched jointly.
Random Search: Sampling Instead of Grid
RandomizedSearchCV samples a fixed number of hyperparameter combinations (controlled by n_iter) randomly from specified distributions rather than testing every grid point. Research by Bergstra and Bengio (2012) showed that for the same computational budget, random search finds better hyperparameters than grid search when only a few hyperparameters strongly influence model performance — because random search effectively explores more distinct values of the important parameters.
from sklearn.svm import SVC
from sklearn.model_selection import RandomizedSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
from scipy.stats import loguniform
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC())])
param_dist = {'svc__C': loguniform(0.01, 100), 'svc__gamma': loguniform(0.0001, 1)}
# 30 random combinations * 5 folds = 150 model fits (same as a 5*6 grid)
rnd = RandomizedSearchCV(pipe, param_dist, n_iter=30, cv=5, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print('Best params:', {k: round(v, 5) for k, v in rnd.best_params_.items()})
print('Best CV score:', round(rnd.best_score_, 4))Why Random Search Wins for Many Hyperparameters
Consider 9 hyperparameters, 3 of which matter and 6 of which are irrelevant. A grid with 3 values per parameter tests 3^9 = 19683 combinations but only tests 3 distinct values for each important parameter. With the same budget of 19,683 random samples, each important parameter is explored with 19,683 distinct values. Concentrating samples in the space of important parameters gives random search a decisive advantage. The probability that at least one random configuration is in the top 5% of the space with just 60 iterations is over 95%.
Distributions for Random Search
Choosing appropriate probability distributions for RandomizedSearchCV is important. Use scipy.stats.loguniform(a, b) for parameters that span orders of magnitude like learning_rate or C. Use scipy.stats.uniform(a, b-a) for parameters with a linear scale like subsample (0.5 to 1.0). Use scipy.stats.randint(low, high) for integer parameters like n_estimators or max_depth. Using list-valued parameters (e.g., [3, 5, 7, 9]) samples uniformly from those discrete options.
from scipy.stats import loguniform, uniform, randint
import numpy as np
# Example distributions for RandomForestClassifier + LogisticRegression pipeline
param_dist = {
'rf__n_estimators': randint(50, 500), # integer, uniform
'rf__max_depth': [3, 5, 7, None], # discrete list
'rf__min_samples_leaf': randint(1, 20), # integer, uniform
'rf__max_features': loguniform(0.1, 1.0) # continuous, log-scale
}
# Show 5 sample combinations
np.random.seed(42)
for _ in range(3):
sample = {k: v.rvs() if hasattr(v, 'rvs') else np.random.choice(v) for k, v in param_dist.items()}
print(sample)Comparing Grid and Random Search Side by Side
A direct comparison: grid search with 5 values for each of 3 parameters requires 125 fits; random search with 125 iterations covers the same budget but explores continuous distributions instead of 5 fixed points per parameter. In practice, for simple 2D searches (C and gamma for SVM), grid search is perfectly adequate. For complex models like gradient boosting with 6+ hyperparameters, random search with 50-100 iterations consistently outperforms a grid with equivalent computation.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV, cross_val_score
from sklearn.datasets import load_breast_cancer
from scipy.stats import randint
import time, numpy as np
X, y = load_breast_cancer(return_X_y=True)
# Grid search
start = time.time()
grid = GridSearchCV(RandomForestClassifier(random_state=42), {'n_estimators': [50,100,200], 'max_depth': [3,5,None]}, cv=3, n_jobs=-1)
grid.fit(X, y)
print(f'Grid search: {round(time.time()-start,1)}s, best={round(grid.best_score_,4)}')
# Random search
start = time.time()
rnd = RandomizedSearchCV(RandomForestClassifier(random_state=42), {'n_estimators': randint(10,300), 'max_depth': [3,5,7,None]}, n_iter=9, cv=3, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print(f'Random search: {round(time.time()-start,1)}s, best={round(rnd.best_score_,4)}')Halving Grid Search for Larger Spaces
scikit-learn 0.24+ introduced HalvingGridSearchCV and HalvingRandomSearchCV based on the successive halving algorithm: start with all candidates trained on a small data subset, eliminate the worst half, double the data, repeat until one winner remains. This finds good hyperparameters with far less computation than full grid or random search, making it practical for larger search spaces with expensive models.
from sklearn.experimental import enable_halving_search_cv # noqa
from sklearn.model_selection import HalvingRandomSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from scipy.stats import randint
X, y = load_breast_cancer(return_X_y=True)
halving = HalvingRandomSearchCV(
RandomForestClassifier(random_state=42),
{'n_estimators': randint(10, 500), 'max_depth': [3, 5, 7, None], 'min_samples_leaf': randint(1, 20)},
cv=3, factor=2, random_state=42, n_jobs=-1
)
halving.fit(X, y)
print('Best params:', halving.best_params_)
print('Best score:', round(halving.best_score_, 4))Bayesian Optimisation: The Smart Alternative
Both grid and random search are uninformed — they do not use results from previous evaluations to guide future choices. Bayesian optimisation builds a probabilistic model of the objective function (the CV score as a function of hyperparameters) and uses it to intelligently select the most promising combination to evaluate next. Libraries like Optuna, BayesSearchCV (scikit-optimize), and HyperOpt implement this and typically find better hyperparameters in far fewer evaluations than random search.
Practical Search Strategy Recommendations
A practical guideline: (1) start with random search for 30-100 iterations to identify the promising hyperparameter region; (2) if you need more precision, run a fine grid search around that region; (3) for expensive models (slow training), use Bayesian optimisation (Optuna) to minimise evaluations; (4) always use a Pipeline to prevent leakage; (5) set n_jobs=-1 for parallelism; (6) use refit=True (default) so the best model is refitted on the full training data after search. Never use the test set to pick hyperparameters.
Accessing All Results from GridSearchCV
After fitting, grid.cv_results_ is a dictionary containing the mean test score, standard deviation, fit times, and parameter values for every combination evaluated. Converting it to a Pandas DataFrame makes it easy to sort, filter, and visualise. This is useful for understanding the sensitivity of the score to each parameter — if all C values produce similar scores but gamma has a big effect, you know future tuning should focus on gamma.
import pandas as pd
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
grid = GridSearchCV(pipe, {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}, cv=5)
grid.fit(X, y)
df = pd.DataFrame(grid.cv_results_)[['param_svc__C', 'param_svc__gamma', 'mean_test_score', 'std_test_score']]
print(df.sort_values('mean_test_score', ascending=False).head(5).round(4))Quick Check
Test your understanding of Grid Search vs Random Search from this lesson.
Lesson Recap
In this lesson you learned: Grid Search evaluates every combination exhaustively but scales exponentially with hyperparameter count, Random Search samples continuously from distributions and wins when few hyperparameters matter, and Halving and Bayesian optimisation further reduce evaluation cost for expensive models. Next up we explore Nested Cross-Validation for simultaneously selecting and evaluating hyperparameters without bias.
常见问题解答
「网格搜索与随机搜索」课时是免费的吗？
是的 — 「网格搜索与随机搜索」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。
「网格搜索与随机搜索」这节课中我会学到什么？
您将在相同的超参数空间上配置 GridSearchCV 和 RandomizedSearchCV，比较它们的覆盖范围与计算成本，并为大型空间选择更快的方法。 你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Machine Learning Academy 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 3 节课，共 4 节。
「网格搜索与随机搜索」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Machine Learning Academy 课中编写并运行代码吗？
能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。