Support Vector Machines

Maximum-margin classifiers using kernel methods for non-linear decision boundaries.

intermediate classical ml supervised-learningclassification
kernel-methodsoptimisationmarginclassificationstatistics

Author @Praneeth-Suresh Updated 2026-06-24

Overview

SVMs find the hyperplane that maximises the margin between classes:

max⁡w,b2∥w∥s.t.yi(w⊤xi+b)≥1\max_{\mathbf{w}, b} \frac{2}{\|\mathbf{w}\|} \quad \text{s.t.} \quad y_i(\mathbf{w}^\top \mathbf{x}_i + b) \ge 1

Kernel Trick

Non-linear boundaries are achieved by mapping inputs to a high-dimensional feature space implicitly via a kernel K(xi,xj)=ϕ(xi)⊤ϕ(xj)K(\mathbf{x}_i, \mathbf{x}_j) = \phi(\mathbf{x}_i)^\top \phi(\mathbf{x}_j).

SVMs extend Linear Regression to classification and share optimisation foundations with Gradient Descent.

Learning Path

What should I learn before Support Vector Machines?

Start with linear-regression before studying Support Vector Machines.

How does Support Vector Machines connect to other AI topics?

Support Vector Machines belongs to Classical ML and connects to supervised-learning, classification, kernel-methods, optimisation, margin, classification, statistics.

Further Reading

  1. A Tutorial on Support Vector Machines – Burges