SVM Theory: Margins and Support Vectors
Maximum margin classifier, hard vs soft margin (C parameter), support vectors.
SVM Theory: Margins and Support Vectors is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What Is an SVM
A Support Vector Machine is a classifier that finds the boundary best separating two classes. Instead of any separating line, it seeks the one with the widest gap between classes.
The Separating Hyperplane
In 2D the boundary is a line; in higher dimensions it is a hyperplane. The SVM defines it as w . x + b = 0, classifying points by which side they fall on.
The Margin
The margin is the distance between the hyperplane and the nearest points of each class. SVMs maximize this margin because a wider gap usually generalizes better to new data.
Maximum Margin Hyperplane
Among all valid boundaries, the SVM picks the maximum margin hyperplane: the one farthest from the closest points on both sides. This is the core idea that makes SVMs robust.
Support Vectors
The support vectors are the data points closest to the boundary, the ones touching the margin edges. Only they determine the hyperplane; removing other points does not change it.
from sklearn.svm import SVC
model = SVC(kernel="linear")
model.fit(X, y)
print("Support vectors:\n", model.support_vectors_)
print("Indices:", model.support_)Why Support Vectors Matter
Because only support vectors define the model, SVMs are memory efficient at prediction time and focus on the hardest, most informative examples near the boundary rather than easy ones far away.
Hard Margin
A hard margin SVM demands perfect separation: no point may cross the margin. This only works when the data is perfectly linearly separable and is very sensitive to outliers.
The Problem with Hard Margins
Real data is noisy and rarely perfectly separable. A single outlier can make a hard margin impossible or force an absurdly thin margin. We need to allow some mistakes.
Soft Margin
A soft margin permits some points to violate the margin or be misclassified, trading a few errors for a wider, more robust boundary. This is controlled by the parameter C.
The C Parameter
C sets the penalty for violations. A larger C punishes errors hard, yielding a smaller margin that fits training data tightly (risk of overfitting). A smaller C allows more slack and a wider margin (more regularization).
from sklearn.svm import SVC
strict = SVC(C=100, kernel="linear") # narrow margin, fits closely
relaxed = SVC(C=0.1, kernel="linear") # wide margin, more tolerantTuning C in Practice
Choose C with cross-validation. Too large overfits; too small underfits. The best value balances margin width against training accuracy for your data.
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC
gs = GridSearchCV(SVC(kernel="linear"), {"C": [0.01, 0.1, 1, 10, 100]}, cv=5)
gs.fit(X, y)
print("Best C:", gs.best_params_)Quick Check
Test your SVM theory.
Recap
Recap: SVMs find the maximum margin hyperplane separating classes. The support vectors are the closest points and alone define the boundary. A hard margin needs perfect separation; a soft margin allows violations controlled by C (larger C = smaller margin, less regularization).
Frequently asked questions
Is the “SVM Theory: Margins and Support Vectors” lesson free?
Yes — the full text of “SVM Theory: Margins and Support Vectors” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “SVM Theory: Margins and Support Vectors”?
Maximum margin classifier, hard vs soft margin (C parameter), support vectors. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “SVM Theory: Margins and Support Vectors” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- SVM Theory: Margins and Support Vectors
- Kernel Trick: RBF, Polynomial, and Sigmoid
- SVMs for Classification with sklearn
- SVMs for Regression (SVR)