Python Coding Classes for Children in Tartu

Chapter 123 – Build a KNN Model with Scikit-Learn

1. End-to-End Pipeline

1. Load data (handwritten digits)
2. Stratified 75/25 train–test split
3. Scale features with StandardScaler
4. Fit KNeighborsClassifier (k=5, Euclidean)
5. Predict on test set and report accuracy
6. Inspect a 10×10 confusion matrix

2. Dataset: load_digits

• ~1,800 samples of digits 0–9
• Each image is 8×8 pixels → 64 features
• stratify=y keeps class balance in train and test

from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

digits = load_digits()
X, y = digits.data, digits.target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)
# ~1347 train, ~450 test

3. Scale and Fit

from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, confusion_matrix

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

knn = KNeighborsClassifier(n_neighbors=5, metric='euclidean')
knn.fit(X_train_s, y_train)
y_pred = knn.predict(X_test_s)
print(accuracy_score(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))

Typical test accuracy ≈ 96.44%. Defaults of note: weights='uniform', algorithm='auto', leaf_size=30, p=2.

4. Confusion Matrix Insights

Strong diagonal dominance means most digits are correct. A common confusion: digit 4 mislabeled as 8 a few times (similar strokes). Fix: add more training examples of those hard pairs so the neighborhood boundary sharpens.

5. Production Reminders

• Always scale before Euclidean/Manhattan KNN.
• Tune k and metric with validation when accuracy plateaus.
• Use the confusion matrix to target data collection, not only overall accuracy.