mBlock STEM Robotics Starter Course for Phuket Expat Kids

1. What Outliers Are

Outliers sit far from the main body of the data. Left alone, they skew summaries, pull regression lines, and confuse models. Detecting and handling them is a standard preprocessing step.

2. Three Handling Strategies

• Clipping — Cap values at a fixed threshold (e.g. 1,000).
• Winsorization — Pull extreme percentiles (e.g. top 20%) to boundary values; keep every row.
• Removal — Drop rows or set extreme values to null; largest reduction in sample size.

3. Detection

Summary stats: Compare Q3 (75%) to the max with .describe(). A large gap (e.g. Q3 = 1,300, max = 5,000) flags upper outliers.

Box plots: The box and whiskers show the bulk of the data; points beyond the whiskers (e.g. 3,000 and 5,000) are visual outliers.

4. Python: Three Methods

import pandas as pd
import numpy as np
from scipy.stats.mstats import winsorize

sales = pd.Series([200, 400, 800, 1000, 1100, 1200, 1300, 1500, 3000, 5000])
df = pd.DataFrame({"Original": sales})
print(df.describe())  # Q3 vs max gap

# Clipping
df["Clipped"] = df["Original"].clip(upper=1000)

# Winsorization (top 20%)
df["Winsorized"] = winsorize(df["Original"], limits=[0.0, 0.20])

# Removal
df["Removed"] = df["Original"].apply(lambda x: x if x <= 1000 else np.nan)
df_dropped = df.dropna(subset=["Removed"])

5. When to Use Which

Method	Mechanism	Best when
Clipping	Cap at a threshold	Few outliers
Winsorize	Pull percentiles inward; keep rows	Many outliers; preserve sample size
Removal	Drop rows or set null	Corrupt / unsalvageable points

Compare side by side with a multi-column box plot before choosing a final policy for the pipeline.