STEM Robotics Workshops for Kids in Liberec

Chapter 120 – K-Nearest Neighbors (KNN) Explained

1. What Is KNN?

K-Nearest Neighbors is a simple supervised algorithm for classification and regression. Its idea is "birds of a feather flock together": a new point is labeled by looking at its closest neighbors in feature space.

• Lazy learner: no fitted equation or weights—training data stays in memory and is queried at prediction time.
• Explainable: every decision can be traced to the actual neighbors used.
• Dual use: majority vote for classes; mean of neighbor targets for continuous values.

2. Advantages and Uses

• Easy to understand; non-parametric (no distribution assumptions).
• Works with mixed numerical and categorical features when distances are defined carefully.
• Examples: handwritten digit recognition; house price estimates from similar nearby properties.

3. Four-Step Workflow

1. Choose k — usually an odd number for classification to avoid ties (e.g. 3, 5, 7).
2. Compute distances from the query to every training point (Euclidean or Manhattan).
3. Sort distances and keep the top-k nearest neighbors.
4. Predict — majority vote (classification) or mean of targets (regression).

4. Distance Metrics

• Euclidean: straight-line distance — √(Σ (xᵢ − yᵢ)²).
• Manhattan: grid path — sum of absolute differences.

5. Concrete Examples

Classification (k=5): Neighbors vote Red, Red, Blue, Red, Blue → Red wins 3–2.

Regression (k=3 house prices): $250k, $270k, $260k → predicted price $260k.

6. Summary

Feature	Detail
Paradigm	Supervised (classification & regression)
Training	Lazy / non-parametric (stores data)
Key hyperparameter	k (often odd for classification)
Distances	Euclidean, Manhattan
Aggregation	Vote (class) / mean (regression)
XAI	Transparent neighbor-based logic