STEM Robotics Workshops for Kids in Famagusta

Chapter 115 – Support Vector Machines (SVM) Explained for Beginners

1. What SVM Does

SVM is a strong supervised algorithm used mainly for classification (and also for regression). Instead of any separating boundary, it finds the optimal hyperplane that maximizes the margin between classes. It handles high-dimensional spaces well and can model non-linear data with kernels.

2. Hyperplanes by Dimension

Dimension	Hyperplane looks like
1D	A point on a line
2D	A straight line (\(w \cdot x + b = 0\))
3D	A flat plane
>3D	A generalized hyperplane

3. Support Vectors and Margin

• Support vectors are the points closest to the boundary. Only they determine the hyperplane’s position and orientation. Removing far-away points does not change the solution.
• Margin is the distance from the hyperplane to those nearest points. SVM maximizes this margin for better generalization and less sensitivity to noise.

4. Math in Brief

• Hyperplane: \(w \cdot x + b = 0\)
• Objective: minimize \(\frac{1}{2}\|w\|^2\) (equivalent to maximizing margin \(2/\|w\|\))
• Constraint: \(y_i (w \cdot x_i + b) \ge 1\) (equality holds for support vectors)

5. High Dimensions and Kernels

SVM works well when there are many features. When data is not linearly separable, kernel functions map it so a curved boundary can still separate the classes.

6. Concept Summary

Concept	Role
Hyperplane	Decision boundary (\(w \cdot x + b = 0\))
Support vectors	Closest points that define the boundary
Margin	Distance to support vectors — maximize it
Weights \(w\)	Feature importance / orientation
Bias \(b\)	Shifts the boundary
Kernels	Allow non-linear separation