Computational Thinking Classes for International School Kids in Bangkok

1. Why Transform Features

After cleaning and scaling, two issues still hurt many models: skewed distributions and extreme outliers. Transformation reshapes continuous columns so they are more stable for learning.

2. Skewness and Log Transform

Right or left tails break the near-normal assumption of many linear methods. Common fixes: log, Box-Cox (positive only), Yeo-Johnson (handles zeros/negatives).

import numpy as np
import pandas as pd

df = pd.read_csv("telecom_churn.csv")
print(df[["Monthly Income", "Monthly Bill"]].describe())

df["Income_Log"] = np.log1p(df["Monthly Income"])
print(df[["Monthly Income", "Income_Log"]].head())

np.log1p computes \(\log(1+x)\), safe when zeros appear. Large incomes compress into a tighter, more centered range.

3. Outliers: Why They Matter

Extreme points pull error-driven models off the main pattern. Two practical treatments keep rows but limit leverage: clipping and winsorization.

4. Clipping

upper_limit = df["Monthly Bill"].quantile(0.99)
df["Bill_Clipped"] = df["Monthly Bill"].clip(upper=upper_limit)
print(df["Bill_Clipped"].max(), df["Bill_Clipped"].min())

Values above the 99th percentile are capped at that threshold (e.g. max drops from ~$35k toward ~$21.7k in the example).

5. Winsorization

from scipy.stats.mstats import winsorize

df["Bill_Winsorized"] = winsorize(
    df["Monthly Bill"],
    limits=[0.01, 0.01]
)
print(df["Bill_Winsorized"].max(), df["Bill_Winsorized"].min())

Bottom/top 1% are replaced by the boundary values—no rows dropped, tails pulled inward.

6. Technique Snapshot

Technique	Targets	Tool
Log	Right skew	np.log1p
Box-Cox / Yeo-Johnson	Skew	SciPy
Clipping	Outliers	Series.clip
Winsorize	Outliers	winsorize