0Pricing
Learn AI with Python · Lesson

SVMs for Classification with sklearn

SVC, LinearSVC, kernel selection, class_weight='balanced', probability=True.

SVMs for Classification with sklearn is a free Learn AI with Python lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

SVMs in scikit-learn

scikit-learn offers several SVM classifiers. The main one is SVC, which supports linear and kernel boundaries through its kernel parameter.

from sklearn.svm import SVC

model = SVC(kernel="rbf", C=1.0, gamma="scale")
model.fit(Xtr, ytr)
print(model.score(Xte, yte))

The Core SVC Parameters

Three parameters drive SVC behavior:

  • kernel the boundary type (linear, rbf, poly)
  • C the error penalty / regularization
  • gamma the kernel width for rbf/poly
from sklearn.svm import SVC

model = SVC(kernel="rbf", C=10, gamma=0.1)

Always Scale First

SVMs are scale-sensitive. Wrap the scaler and SVC in a pipeline so scaling is fit only on training data, even inside cross-validation.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

pipe = make_pipeline(StandardScaler(), SVC(kernel="rbf"))
pipe.fit(Xtr, ytr)

LinearSVC for Large Datasets

SVC(kernel="linear") scales poorly to many samples. LinearSVC uses a faster solver optimized for linear problems and handles large, high-dimensional data far better.

from sklearn.svm import LinearSVC

model = LinearSVC(C=1.0, max_iter=5000)
model.fit(Xtr, ytr)

SVC vs LinearSVC

Use LinearSVC for large or sparse data (like text). Use SVC when you need kernels for non-linear boundaries. Note LinearSVC uses a slightly different loss and lacks predict_proba.

Class Imbalance

When one class is rare, the SVM tends to favor the majority. Setting class_weight="balanced" automatically weights classes inversely to their frequency.

from sklearn.svm import SVC

model = SVC(kernel="rbf", class_weight="balanced")
model.fit(Xtr, ytr)

Custom Class Weights

You can also pass an explicit dictionary to class_weight to emphasize a specific class more heavily than the automatic balancing.

from sklearn.svm import SVC

model = SVC(class_weight={0: 1, 1: 5})  # class 1 errors cost 5x

Probability Estimates

By default SVC returns hard labels. Set probability=True to enable predict_proba (via internal calibration), needed for ROC AUC and threshold tuning. It slows training.

from sklearn.svm import SVC

model = SVC(probability=True)
model.fit(Xtr, ytr)
proba = model.predict_proba(Xte)[:, 1]

Multiclass SVMs

SVC handles more than two classes using a one-vs-one scheme automatically. You do not change your code; just fit with multiclass labels and it works.

from sklearn.svm import SVC

model = SVC(decision_function_shape="ovr")
model.fit(X_multiclass, y_multiclass)

Tuning C and gamma with GridSearchCV

The standard recipe is a grid search over C and gamma on a log scale, all inside a scaling pipeline.

from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

pipe = make_pipeline(StandardScaler(), SVC(kernel="rbf"))
param = {
    "svc__C": [0.1, 1, 10, 100],
    "svc__gamma": [0.001, 0.01, 0.1, 1],
}
gs = GridSearchCV(pipe, param, cv=5, scoring="f1")
gs.fit(Xtr, ytr)
print(gs.best_params_)

Practical Tips

SVMs shine on small-to-medium datasets with clear margins. They struggle on very large data (use LinearSVC or switch to boosting). Always scale, tune C and gamma, and use class_weight for imbalance.

Quick Check

Test your sklearn SVM knowledge.

Recap

Recap: Use SVC with kernel, C, and gamma for non-linear classification, always inside a scaling pipeline. Switch to LinearSVC for large/text data. Handle imbalance with class_weight="balanced", and tune C and gamma jointly with GridSearchCV.

Frequently asked questions

Is the “SVMs for Classification with sklearn” lesson free?

Yes — the full text of “SVMs for Classification with sklearn” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “SVMs for Classification with sklearn”?

SVC, LinearSVC, kernel selection, class_weight='balanced', probability=True. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “SVMs for Classification with sklearn” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. SVM Theory: Margins and Support Vectors
  2. Kernel Trick: RBF, Polynomial, and Sigmoid
  3. SVMs for Classification with sklearn
  4. SVMs for Regression (SVR)
← Back to Learn AI with Python