Online STEM Education for Kids in Strasbourg

Chapter 130 – K-Means Inertia and Within-Cluster Variation

1. Objective Function

K-Means groups unlabeled points into K clusters by minimizing intra-cluster variance: the total squared distance from each point to its cluster centroid. That sum is called inertia (also WCSS — within-cluster sum of squares).

Inertia = Σ Σ ||x_i − μ_k||². Squaring removes sign and penalizes large distances more strongly.

2. Low vs High Inertia

• Low inertia: Points sit close to their centroids — tight, cohesive groups.
• High inertia: Points are scattered far from centroids — weak grouping quality.

Training repeatedly shifts centroids to drive inertia down until it stabilizes (convergence) or a max-iteration limit is hit.

3. Outliers and Local Minima

Squared distances make inertia sensitive to outliers, which can pull centroids away from the true cluster core. Random centroid starts can also trap the run in a local minimum. Scikit-Learn’s n_init=10 runs K-Means several times with different seeds and keeps the run with the lowest inertia.

4. Python Sketch

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

X, _ = make_blobs(n_samples=200, centers=3, cluster_std=1.0, random_state=42)
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
labels = kmeans.fit_predict(X)
centroids = kmeans.cluster_centers_
inertia_val = kmeans.inertia_
print(f"Inertia (K=3): {inertia_val:.2f}")

plt.scatter(X[:, 0], X[:, 1], c=labels, cmap="viridis", alpha=0.8, edgecolors="k")
plt.scatter(centroids[:, 0], centroids[:, 1], s=200, c="black", marker="x", linewidths=3)
plt.title(f"K-Means (K=3) | Inertia: {inertia_val:.2f}")
plt.show()

for k in [2, 3, 4, 5]:
    m = KMeans(n_clusters=k, random_state=42, n_init=10).fit(X)
    print(f"K={k} | Inertia: {m.inertia_:.2f}")

5. Demo Numbers

On 200-sample 3-center blobs:

• K=2 → ~3720 (under-clustered)
• K=3 → ~364 (sharp drop; natural fit)
• K=4 → ~314 (small gain)
• K=5 → ~274 (further small gain; overfit risk)

Past the natural K, more clusters always lower inertia a bit but mainly split real groups — diminishing returns and overfitting.

6. Concept Table

Term	Meaning	In Scikit-Learn
Inertia (WCSS)	Sum of squared point–centroid distances	kmeans.inertia_
Centroids	Mean of each cluster	kmeans.cluster_centers_
Convergence	Inertia stops falling	Handled inside .fit()
n_init	Multiple random starts; keep best inertia	KMeans(n_init=10)