Scratch Coding Classes for Kids in Paphos

Chapter 114 – Build a Random Forest Model with Scikit-Learn

1. End-to-End Workflow

1. Load data and separate features / target
2. Stratified train–test split (e.g. 75% / 25%)
3. Fit a baseline RandomForestClassifier
4. Tune key hyperparameters (e.g. n_estimators)
5. Evaluate with accuracy, confusion matrix, precision/recall/F1, and feature importance

2. Core Metrics

• Precision = TP / (TP + FP) — prioritize when false positives are costly (e.g. blocking real users)
• Recall = TP / (TP + FN) — prioritize when missing positives is dangerous (e.g. disease detection)
• F1 = harmonic mean of precision and recall — balanced view on imbalanced data
• Support — count of true samples per class

3. Iris Pipeline (75/25 Stratified Split)

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, confusion_matrix
import numpy as np

iris = load_iris()
X, y = iris.data, iris.target
feature_names = iris.feature_names

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)
# ~112 train, ~38 test

rf_baseline = RandomForestClassifier(
    n_estimators=100, max_depth=None, random_state=42
)
rf_baseline.fit(X_train, y_train)
y_pred = rf_baseline.predict(X_test)
print(f"Baseline accuracy: {accuracy_score(y_test, y_pred) * 100:.2f}%")
print(confusion_matrix(y_test, y_pred))

Typical baseline: ~92.11% accuracy; only a few mix-ups between Versicolor and Virginica (Setosa often perfect).

4. Tuning n_estimators

Trying 20 / 50 / 100 / 200 trees often peaks at 50 and 100 (~92.11%). Prefer the smaller model (50 trees) when accuracy is equal — lower memory and faster inference.

5. Feature Importance

importances = rf_baseline.feature_importances_
indices = np.argsort(importances)[::-1]
for f in range(X.shape[1]):
    i = indices[f]
    print(f"{f+1}. {feature_names[i]}: {importances[i]:.4f}")

Typical ranking: Petal width and petal length dominate (~0.45 each, ~90% combined); sepal length moderate; sepal width low.

6. Why Forest Beats a Single Tree

One tree is readable but sensitive to bias and noise. A forest of diverse bootstrapped trees with random features averages out mistakes and is a strong classical default in practice.