AI Courses for Kids in Narva

Chapter 121 – KNN Distance Metrics Explained

1. Why Distance Matters

KNN decides neighbors by proximity. The metric you choose changes which points count as "near" and thus affects accuracy. Three common metrics: Euclidean, Manhattan, and Minkowski.

2. Euclidean Distance

Straight-line distance in feature space:

d(X, Y) = √(Σ (xᵢ − yᵢ)²)

• Very sensitive to feature scale — always scale or normalize first.
• Best for continuous features (age, weight, height, income).

3. Manhattan Distance

City-block / grid path — sum of absolute differences:

d(X, Y) = Σ |xᵢ − yᵢ|

• Less sensitive to outliers (no squaring of large gaps).
• Useful for discrete or grid-like problems and high-dimensional sparse data.

4. Minkowski Distance

General form controlled by p:

d(X, Y) = (Σ |xᵢ − yᵢ|^p)^(1/p)

• p = 1 → Manhattan
• p = 2 → Euclidean
• p ≥ 3 → more weight on large coordinate differences

5. Numerical Example: (2, 3) to (5, 7)

• Δx = 3, Δy = 4
• Euclidean: √(9 + 16) = 5.0
• Manhattan: 3 + 4 = 7.0
• Minkowski (p=3): (27 + 64)^(1/3) ≈ 4.498

6. Scikit-Learn Comparison

from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X, y = make_classification(n_samples=1000, n_features=2,
                           n_informative=2, n_redundant=0, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42)

for name, kwargs in [
    ('Euclidean', dict(metric='euclidean')),
    ('Manhattan', dict(metric='manhattan')),
    ('Minkowski p=3', dict(metric='minkowski', p=3))
]:
    clf = KNeighborsClassifier(n_neighbors=5, **kwargs)
    clf.fit(X_train, y_train)
    print(name, accuracy_score(y_test, clf.predict(X_test)))

Example results on this split: Manhattan ~94.0%, Euclidean ~92.4%, Minkowski (p=3) ~92.4%. Always try more than one metric and scale features when using Euclidean or Minkowski with p=2.

7. Quick Reference

Metric	Geometry	Notes
Euclidean (p=2)	Straight line	Scale-sensitive; continuous features
Manhattan (p=1)	Grid path	More robust to outliers
Minkowski	Tunable via p	Unifies the two; p≥3 stresses large gaps