Regolare C e gamma con una grid search
Eseguirà una GridSearchCV su C e gamma, interpreterà la heat map di validazione e selezionerà gli iperparametri migliori per un classificatore SVM.
Regolare C e gamma con una grid search è una lezione Machine Learning Academy gratuita su CoddyKit. Questa è la lezione 4 di 4. Puoi leggere la lezione completa qui gratuitamente — poi esercitati direttamente nel browser con un editor di codice integrato e un tutor IA disponibile 24/7. Fa parte del percorso di apprendimento Machine Learning Academy, e i tuoi progressi si sincronizzano tra il web e l'app CoddyKit. Il corso Machine Learning Academy include 4 lezioni in totale.
Parti di questa lezione non sono ancora state tradotte e vengono mostrate in inglese.
Why C and Gamma Must Be Tuned Together
For an RBF SVM, both C and gamma must be tuned simultaneously because their effects interact. A high C with a small gamma may produce a good boundary, but so might a low C with a high gamma. Searching them independently — fixing one while optimising the other — misses this interaction and often settles on a sub-optimal combination. The standard approach is a 2D grid search that evaluates every combination of C and gamma values and picks the pair with the best cross-validated score.
Setting Up the Grid Search
Use GridSearchCV from scikit-learn to evaluate every hyperparameter combination. Wrap your scaler and SVM in a Pipeline so that GridSearchCV refits the scaler on each training fold without leaking test data statistics. Specify the hyperparameter grid using the double-underscore notation svc__C and svc__gamma to target the named step in the pipeline.
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('scaler', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-2, 3, 6), 'svc__gamma': np.logspace(-4, 0, 5)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best CV score:', round(grid.best_score_, 4))Logarithmic Scale for C and Gamma
Both C and gamma should be searched on a logarithmic scale (powers of 10 or powers of 2). The reason is that the model's behaviour changes dramatically as these parameters move from 0.001 to 0.01 (a 10x change), but barely changes from 10 to 11 (a 10% change). A typical coarse grid spans: C in [0.01, 0.1, 1, 10, 100, 1000] and gamma in [0.0001, 0.001, 0.01, 0.1, 1]. After finding the best region, you can refine with a finer grid around those values.
import numpy as np
# Generate logarithmically spaced values
C_range = np.logspace(-2, 3, 6) # [0.01, 0.1, 1, 10, 100, 1000]
gamma_range = np.logspace(-4, 0, 5) # [0.0001, 0.001, 0.01, 0.1, 1.0]
print('C values:', np.round(C_range, 4))
print('gamma values:', np.round(gamma_range, 5))
print('Total combinations:', len(C_range) * len(gamma_range))Interpreting the Validation Heat Map
After a grid search, you can visualise results as a 2D heat map with C on one axis and gamma on the other, coloured by cross-validation score. This reveals: (1) a sweet spot region where scores peak; (2) underfitting at low C and low gamma (too smooth); (3) overfitting at high C and high gamma (too complex). Identifying the shape of the sweet spot helps you understand the model's sensitivity and decide whether to run a finer search.
import numpy as np
import pandas as pd
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X, y)
results = pd.DataFrame(grid.cv_results_)
pivot = results.pivot_table(index='param_svc__C', columns='param_svc__gamma', values='mean_test_score')
print(pivot.round(3))Coarse-to-Fine Grid Search
A practical strategy is coarse-to-fine search: first run a coarse grid over a wide range to identify the promising region, then run a fine grid focused on that region. For example, if C=10 and gamma=0.01 looks best on the coarse grid, search C in [5, 8, 10, 15, 20] and gamma in [0.005, 0.008, 0.01, 0.015, 0.02] for the fine grid. This reduces total computation compared to running a fine grid over the entire space from the start.
RandomizedSearchCV as an Alternative
For larger grids or when the optimal region is unknown, RandomizedSearchCV samples random combinations from continuous distributions rather than testing every grid point. This scales much better: evaluating 50 random combinations from a 10×10 grid (100 points) finds the optimum nearly as reliably as testing all 100 while using half the compute. Use scipy.stats.loguniform for C and gamma distributions to preserve the logarithmic sampling behaviour.
from sklearn.svm import SVC
from sklearn.model_selection import RandomizedSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
from scipy.stats import loguniform
X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_dist = {'svc__C': loguniform(0.01, 1000), 'svc__gamma': loguniform(0.0001, 1)}
rnd = RandomizedSearchCV(pipe, param_dist, n_iter=50, cv=5, random_state=42, n_jobs=-1)
rnd.fit(X, y)
print('Best params:', {k: round(v, 5) for k, v in rnd.best_params_.items()})
print('Best CV score:', round(rnd.best_score_, 4))Evaluating the Best Model on the Test Set
After grid search, GridSearchCV.best_estimator_ is automatically refitted on the entire training set with the best hyperparameters. Evaluate it on the held-out test set using grid.score(X_test, y_test) — call this only once, at the very end. If you use the test score to continue tuning, you are effectively leaking test set information into the model selection process and will get an overly optimistic performance estimate.
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
pipe = Pipeline([('sc', StandardScaler()), ('svc', SVC(kernel='rbf'))])
param_grid = {'svc__C': np.logspace(-1, 2, 4), 'svc__gamma': np.logspace(-3, 0, 4)}
grid = GridSearchCV(pipe, param_grid, cv=5, n_jobs=-1)
grid.fit(X_train, y_train)
print('CV score:', round(grid.best_score_, 4))
print('Test score:', round(grid.score(X_test, y_test), 4))Computation Cost of Grid Search
A grid search with k folds, m C values, and n gamma values trains k × m × n models. A 5-fold search over a 6×5 grid requires 150 model fits. Each fit on a dataset with 1,000 examples and an RBF SVM takes a fraction of a second, but on 100,000 examples it can take minutes per fit. To manage cost: use n_jobs=-1 for parallelism, start with coarse grids, and consider RandomizedSearchCV with fewer iterations for large datasets or expensive models.
Nested Cross-Validation for Unbiased Estimates
Selecting hyperparameters based on validation scores and then evaluating on a separate test set can still give slightly optimistic estimates due to the number of hyperparameter combinations tried. Nested cross-validation uses an outer CV loop for evaluation and an inner CV loop (grid search) for selection. The outer fold test set is never used in any model fitting or hyperparameter selection, giving a truly unbiased generalisation estimate. This is the gold standard for small datasets where you cannot afford a dedicated test set.
Practical Tips for SVM Tuning
Practical advice for SVM grid search: (1) always include your scaler in the pipeline to prevent leakage; (2) use gamma='scale' as the starting point before exploring fixed values; (3) if the best score is at the edge of your grid (e.g., C=1000 is best), extend the grid further; (4) compare the CV score and test score — a large gap indicates overfitting during hyperparameter selection; (5) for multi-class problems, SVM extends automatically using one-vs-one or one-vs-rest strategies under the hood.
SVM Predict_proba via Platt Scaling
By default, SVC does not produce probability estimates — only hard class labels or raw decision function scores. Setting probability=True enables Platt scaling: an additional logistic regression model is fitted on top of the SVM's decision function scores using cross-validation to calibrate the outputs to probabilities. This makes the SVM 5-10x slower to train and slightly changes the decision boundary. Use probability estimates only when you need them (soft voting, ROC-AUC, calibration) and stick to hard predictions otherwise.
from sklearn.svm import SVC
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = make_pipeline(StandardScaler(), SVC(kernel='rbf', C=10, gamma=0.01, probability=True))
model.fit(X_train, y_train)
probs = model.predict_proba(X_test)
print('Probability shape:', probs.shape)
print('First 3 predicted probs (class 0, class 1):'); print(probs[:3].round(3))Quick Check
Test your understanding of Grid Search for SVM hyperparameters from this lesson.
Lesson Recap
In this lesson you learned: C and gamma must be tuned together with a 2D grid search, both should be searched on a logarithmic scale to cover relevant magnitudes, and always include the scaler in the pipeline to prevent data leakage during cross-validation. Next up we explore Gradient Boosting and how sequential error correction builds powerful ensembles.
Impara Python con un tutor IA — gratis
Scrivi ed esegui vero codice nel tuo browser, ricevi aiuto istantaneo da un tutor IA disponibile 24/7, e riprendi da dove hai lasciato sul web o nell'app.
- Corsi
- 30
- Lezioni
- 120
Domande Frequenti
La lezione «Regolare C e gamma con una grid search» è gratuita?
Sì — il testo completo di «Regolare C e gamma con una grid search» è gratuito qui sul web. Per esercitarvi in modo interattivo (un editor di codice integrato e un tutor IA 24/7) e sbloccare il resto del corso Machine Learning Academy, passa a CoddyKit PRO. Il corso Machine Learning Academy include 4 lezioni in totale.
Cosa imparerò in «Regolare C e gamma con una grid search»?
Eseguirà una GridSearchCV su C e gamma, interpreterà la heat map di validazione e selezionerà gli iperparametri migliori per un classificatore SVM. Eserciti Machine Learning Academy con codice pratico che esegui direttamente nel browser, e un tutor IA 24/7 risponde alle tue domande mentre lavori sulla lezione.
Ho bisogno di esperienza per iniziare Machine Learning Academy?
Non è richiesta alcuna esperienza precedente. Machine Learning Academy su CoddyKit è strutturato per principianti e studenti avanzati, quindi puoi iniziare da qui o dall'inizio e procedere al tuo ritmo. Questa è la lezione 4 di 4.
Quanto tempo richiede la lezione «Regolare C e gamma con una grid search»?
La maggior parte delle lezioni CoddyKit richiede circa 5–10 minuti. Ogni lezione è breve e interattiva, quindi fai progressi costanti e riprendi esattamente da dove hai lasciato su web e app.
Posso scrivere ed eseguire codice in questa lezione Machine Learning Academy?
Sì. Ogni lezione Machine Learning Academy include un editor di codice integrato, quindi scrivi ed esegui codice reale direttamente nel tuo browser e ricevi feedback istantaneo dall'IA — nessuna configurazione locale necessaria.
Tutte le lezioni di questo corso
- Classificatore a margine massimo: vettori di supporto e iperpiano
- SVM a margine morbido e parametro C
- Il kernel trick: kernel RBF, polinomiale e sigmoide
- Regolare C e gamma con una grid search