Maximum Margin Classifier: Support Vectors and Hyperplane

Support Vector Machines (SVMs) are classifiers that find the best separating boundary between two classes. When multiple boundaries can separate the classes, which one should you choose? The SVM's answer is elegant: pick the boundary that is as far as possible from every training example. This maximally distant boundary is called the maximum-margin hyperplane, and SVM's theoretical foundations guarantee that it generalises better to unseen data than arbitrary separating boundaries.

In 2D, a hyperplane is a line (1 dimension less than the data space). In 3D, it is a plane. In general p-dimensional space, it is a (p-1)-dimensional flat surface defined by the equation w·x + b = 0, where w is the normal vector (perpendicular to the surface), x is the input feature vector, and b is the bias term. Points on one side satisfy w·x + b > 0 (predicted positive class) and points on the other satisfy w·x + b < 0 (predicted negative class).