Kernel Trick: RBF, Polynomial, and Sigmoid
Why kernels work, RBF kernel gamma parameter, choosing kernels for different data shapes.
Kernel Trick: RBF, Polynomial, and Sigmoid is a free Learn AI with Python lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
When Lines Are Not Enough
A linear SVM only draws straight boundaries. Many datasets are not linearly separable, a circle of one class surrounded by another cannot be split by a line. The kernel trick solves this.
Mapping to Higher Dimensions
If data is not separable in its original space, projecting it into a higher-dimensional space can make it separable. A curve in 2D can become a flat plane in 3D.
The Kernel Trick
The kernel trick computes the similarity K(x, z) between points as if they were mapped to high dimensions, without ever doing the expensive mapping. This makes non-linear SVMs efficient.
What a Kernel Function Is
A kernel K(x, z) is a function returning the dot product of x and z in some high-dimensional feature space. Different kernels imply different feature mappings and boundary shapes.
The RBF Kernel
The RBF (Gaussian) kernel is the most popular default: K(x, z) = exp(-gamma * ||x - z||^2). It measures similarity that decays with distance, allowing flexible, curved boundaries.
from sklearn.svm import SVC
model = SVC(kernel="rbf", gamma="scale", C=1.0)
model.fit(X, y)The gamma Parameter
gamma controls the RBF width. Large gamma means each point has a narrow influence, giving a wiggly boundary that can overfit. Small gamma means wide influence and a smoother boundary.
from sklearn.svm import SVC
wiggly = SVC(kernel="rbf", gamma=10) # tight, can overfit
smooth = SVC(kernel="rbf", gamma=0.01) # broad, may underfitgamma and C Together
RBF SVMs depend on both gamma (boundary flexibility) and C (error tolerance). They interact, so you usually tune them jointly with grid search.
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC
param = {"C": [0.1, 1, 10], "gamma": [0.01, 0.1, 1]}
gs = GridSearchCV(SVC(kernel="rbf"), param, cv=5)
gs.fit(X, y)The Polynomial Kernel
The polynomial kernel K(x, z) = (gamma * x . z + r)^degree models interactions up to a chosen degree. Use it when you expect feature interactions of a known order.
from sklearn.svm import SVC
model = SVC(kernel="poly", degree=3, coef0=1, gamma="scale")
model.fit(X, y)The Sigmoid Kernel
The sigmoid kernel K(x, z) = tanh(gamma * x . z + r) behaves like a neural network activation. It is less common and not always a valid kernel, so it is rarely the first choice.
from sklearn.svm import SVC
model = SVC(kernel="sigmoid", gamma="scale", coef0=0)Choosing a Kernel
Guidance:
- Linear for high-dimensional or text data
- RBF as a strong general default for non-linear data
- Polynomial when interactions of a known degree matter
- Sigmoid rarely, niche cases
Always Scale Features
Kernels like RBF rely on distances, so feature scale matters enormously. Always standardize features before using a kernel SVM, or large-scale features will dominate the similarity.
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
pipe = make_pipeline(StandardScaler(), SVC(kernel="rbf"))
pipe.fit(X, y)Quick Check
Test your kernel knowledge.
Recap
Recap: The kernel trick computes K(x, z) as a high-dimensional dot product without the mapping cost, enabling non-linear boundaries. RBF uses exp(-gamma * ||x-z||^2) where large gamma overfits. Polynomial captures degree-d interactions. Always scale features before kernel SVMs.
Frequently asked questions
Is the “Kernel Trick: RBF, Polynomial, and Sigmoid” lesson free?
Yes — the full text of “Kernel Trick: RBF, Polynomial, and Sigmoid” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Kernel Trick: RBF, Polynomial, and Sigmoid”?
Why kernels work, RBF kernel gamma parameter, choosing kernels for different data shapes. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Kernel Trick: RBF, Polynomial, and Sigmoid” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- SVM Theory: Margins and Support Vectors
- Kernel Trick: RBF, Polynomial, and Sigmoid
- SVMs for Classification with sklearn
- SVMs for Regression (SVR)