AI Courses for Kids in Ostrava

Chapter 116 – SVM Margin Explained

1. Goal: Maximize the Margin

SVM places a hyperplane so that the gap (margin) to the nearest points of each class is as large as possible. That gap is a safety zone that improves generalization on new data.

2. Boundary, Support Vectors, and Geometry

• Hyperplane (2D): line \(w \cdot x + b = 0\). One side → Class 0; other side → Class 1.
• Support vectors: the closest points to that line. Only they fix the final boundary; other points can move without changing it.
• Margin: \(\frac{1}{\|w\|}\) — distance from the decision line to the parallel margin lines at \(w \cdot x + b = \pm 1\).

3. Hard vs Soft Margin (C)

• High C (e.g. 10): hard margin — penalizes violations heavily; tight fit around support vectors.
• Low C (e.g. 0.1): soft margin — allows a few violations for a wider, more robust gap.

4. Python Sketch (Linear SVC + Plot)

from sklearn.svm import SVC
from sklearn.datasets import make_blobs
import numpy as np
import matplotlib.pyplot as plt

X, y = make_blobs(n_samples=100, centers=2, cluster_std=1.1, random_state=42)
svm = SVC(kernel='linear', C=10)
svm.fit(X, y)

w, b = svm.coef_[0], svm.intercept_[0]
margin = 1.0 / np.linalg.norm(w)
print("Margin:", margin, "| Support vectors:", len(svm.support_vectors_))

# Plot decision line and parallel margins, highlight support vectors

Typical run: a few support vectors (often one per class on clean blobs), margin width from \(1/\|w\|\), decision line and two parallel dotted margins.

5. Parameter Cheat Sheet

Item	Role
C	Hard vs soft margin trade-off
kernel='linear'	Linear separator
svm.coef_ / intercept_	\(w\) and \(b\)
1 / ||w||	Margin width
support_vectors_	Points that define the boundary