Machine Learning Academy · レッスン

Grid SearchによるCとGammaの調整

CとgammaについてGridSearchCVを実行し、検証ヒートマップを解釈して、SVM分類器に最適なハイパーパラメータを選択します。

「Grid SearchによるCとGammaの調整」はCoddyKit上の無料Machine Learning Academyレッスンです。これはレッスン4/4です。下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。これはMachine Learning Academy学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Machine Learning Academyコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

Why C and Gamma Must Be Tuned Together

For an RBF SVM, both C and gamma must be tuned simultaneously because their effects interact. A high C with a small gamma may produce a good boundary, but so might a low C with a high gamma. Searching them independently — fixing one while optimising the other — misses this interaction and often settles on a sub-optimal combination. The standard approach is a 2D grid search that evaluates every combination of C and gamma values and picks the pair with the best cross-validated score.

Setting Up the Grid Search

Use GridSearchCV from scikit-learn to evaluate every hyperparameter combination. Wrap your scaler and SVM in a Pipeline so that GridSearchCV refits the scaler on each training fold without leaking test data statistics. Specify the hyperparameter grid using the double-underscore notation svc__C and svc__gamma to target the named step in the pipeline.

from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('scaler', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-2, 3, 6), 'svc__gamma': np.logspace(-4, 0, 5)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best CV score:', round(grid.best_score_, 4))

Logarithmic Scale for C and Gamma

Both C and gamma should be searched on a logarithmic scale (powers of 10 or powers of 2). The reason is that the model's behaviour changes dramatically as these parameters move from 0.001 to 0.01 (a 10x change), but barely changes from 10 to 11 (a 10% change). A typical coarse grid spans: C in [0.01, 0.1, 1, 10, 100, 1000] and gamma in [0.0001, 0.001, 0.01, 0.1, 1]. After finding the best region, you can refine with a finer grid around those values.

import numpy as np
# Generate logarithmically spaced values
C_range = np.logspace(-2, 3, 6)       # [0.01, 0.1, 1, 10, 100, 1000]
gamma_range = np.logspace(-4, 0, 5)   # [0.0001, 0.001, 0.01, 0.1, 1.0]
print('C values:', np.round(C_range, 4))
print('gamma values:', np.round(gamma_range, 5))
print('Total combinations:', len(C_range) * len(gamma_range))

Interpreting the Validation Heat Map

After a grid search, you can visualise results as a 2D heat map with C on one axis and gamma on the other, coloured by cross-validation score. This reveals: (1) a sweet spot region where scores peak; (2) underfitting at low C and low gamma (too smooth); (3) overfitting at high C and high gamma (too complex). Identifying the shape of the sweet spot helps you understand the model's sensitivity and decide whether to run a finer search.

import numpy as np
import pandas as pd
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
results = pd.DataFrame(grid.cv_results_)
pivot = results.pivot_table(index='param_svc__C', columns='param_svc__gamma', values='mean_test_score')
print(pivot.round(3))

Coarse-to-Fine Grid Search

A practical strategy is coarse-to-fine search: first run a coarse grid over a wide range to identify the promising region, then run a fine grid focused on that region. For example, if C=10 and gamma=0.01 looks best on the coarse grid, search C in [5, 8, 10, 15, 20] and gamma in [0.005, 0.008, 0.01, 0.015, 0.02] for the fine grid. This reduces total computation compared to running a fine grid over the entire space from the start.

RandomizedSearchCV as an Alternative

For larger grids or when the optimal region is unknown, RandomizedSearchCV samples random combinations from continuous distributions rather than testing every grid point. This scales much better: evaluating 50 random combinations from a 10×10 grid (100 points) finds the optimum nearly as reliably as testing all 100 while using half the compute. Use scipy.stats.loguniform for C and gamma distributions to preserve the logarithmic sampling behaviour.

from sklearn.svm import SVC
from sklearn.model_selection import RandomizedSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
from scipy.stats import loguniform

X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_dist = {'svc__C': loguniform(0.01, 1000), 'svc__gamma': loguniform(0.0001, 1)}
rnd = RandomizedSearchCV(pipe, param_dist, n_iter=50, cv=5, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print('Best params:', {k: round(v, 5) for k, v in rnd.best_params_.items()})
print('Best CV score:', round(rnd.best_score_, 4))

Evaluating the Best Model on the Test Set

After grid search, GridSearchCV.best_estimator_ is automatically refitted on the entire training set with the best hyperparameters. Evaluate it on the held-out test set using grid.score(X_test, y_test) — call this only once, at the very end. If you use the test score to continue tuning, you are effectively leaking test set information into the model selection process and will get an overly optimistic performance estimate.

from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X_train, y_train)
print('CV score:', round(grid.best_score_, 4))
print('Test score:', round(grid.score(X_test, y_test), 4))

Computation Cost of Grid Search

A grid search with k folds, m C values, and n gamma values trains k × m × n models. A 5-fold search over a 6×5 grid requires 150 model fits. Each fit on a dataset with 1,000 examples and an RBF SVM takes a fraction of a second, but on 100,000 examples it can take minutes per fit. To manage cost: use n_jobs=-1 for parallelism, start with coarse grids, and consider RandomizedSearchCV with fewer iterations for large datasets or expensive models.

Nested Cross-Validation for Unbiased Estimates

Selecting hyperparameters based on validation scores and then evaluating on a separate test set can still give slightly optimistic estimates due to the number of hyperparameter combinations tried. Nested cross-validation uses an outer CV loop for evaluation and an inner CV loop (grid search) for selection. The outer fold test set is never used in any model fitting or hyperparameter selection, giving a truly unbiased generalisation estimate. This is the gold standard for small datasets where you cannot afford a dedicated test set.

Practical Tips for SVM Tuning

Practical advice for SVM grid search: (1) always include your scaler in the pipeline to prevent leakage; (2) use gamma='scale' as the starting point before exploring fixed values; (3) if the best score is at the edge of your grid (e.g., C=1000 is best), extend the grid further; (4) compare the CV score and test score — a large gap indicates overfitting during hyperparameter selection; (5) for multi-class problems, SVM extends automatically using one-vs-one or one-vs-rest strategies under the hood.

SVM Predict_proba via Platt Scaling

By default, SVC does not produce probability estimates — only hard class labels or raw decision function scores. Setting probability=True enables Platt scaling: an additional logistic regression model is fitted on top of the SVM's decision function scores using cross-validation to calibrate the outputs to probabilities. This makes the SVM 5-10x slower to train and slightly changes the decision boundary. Use probability estimates only when you need them (soft voting, ROC-AUC, calibration) and stick to hard predictions otherwise.

from sklearn.svm import SVC
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = make_pipeline(StandardScaler(), SVC(kernel='rbf', C=10, gamma=0.01, probability=True))
model.fit(X_train, y_train)
probs = model.predict_proba(X_test)
print('Probability shape:', probs.shape)
print('First 3 predicted probs (class 0, class 1):'); print(probs[:3].round(3))

Quick Check

Test your understanding of Grid Search for SVM hyperparameters from this lesson.

Lesson Recap

In this lesson you learned: C and gamma must be tuned together with a 2D grid search, both should be searched on a logarithmic scale to cover relevant magnitudes, and always include the scaler in the pipeline to prevent data leakage during cross-validation. Next up we explore Gradient Boosting and how sequential error correction builds powerful ensembles.

よくある質問

「Grid SearchによるCとGammaの調整」レッスンは無料ですか？

はい。「Grid SearchによるCとGammaの調整」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Machine Learning Academyコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Machine Learning Academyコースには全4レッスンが含まれています。

「Grid SearchによるCとGammaの調整」で何を学びますか？

CとgammaについてGridSearchCVを実行し、検証ヒートマップを解釈して、SVM分類器に最適なハイパーパラメータを選択します。ブラウザで直接実行するハンズオンコードでMachine Learning Academyを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Machine Learning Academyを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのMachine Learning Academyは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。これはレッスン4/4です。

「Grid SearchによるCとGammaの調整」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このMachine Learning Academyレッスンでコードを書いて実行できますか？

はい。すべてのMachine Learning Academyレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

← Machine Learning Academyに戻る