AI Courses for Kids in Larnaca

Chapter 111 – How Random Forest Works — Part 2

1. Tree Aggregation & Feature Diversity

Part 1 covered bootstrapping and bagging. Part 2 focuses on how trees use different feature subsets and how their predictions are combined into a stronger result.

Random feature selection: each tree sees only a random subset of features (e.g. Tree 1: Income + Age; Tree 2: Credit Score + Debt). Across a forest of ~50 trees, no single feature can dominate every split, which reduces feature bias.

Combining many weak trees trained on different data and feature subsets yields one robust strong classifier.

2. Prediction Mechanisms

• Classification — majority voting: each tree votes for a class; the majority wins and individual errors are overruled by consensus.
• Regression — averaging: the mean of all tree outputs is the final value. Example: predictions $300k, $310k, $290k, $305k → average $300k.

3. Advantages of Aggregation

• Higher accuracy: one tree might reach ~78%; a 100-tree forest often approaches ~90%.
• Robustness to noise/outliers: extreme points affect few trees and are averaged out.
• High-dimensional data: features are split across trees instead of one huge tree.
• Feature importance: usage across trees (and OOB checks) ranks which variables matter most.

4. Python: Breast Cancer Classification

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

data = load_breast_cancer()
X, y = data.data, data.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42
)

rf = RandomForestClassifier(
    n_estimators=100,
    max_depth=None,
    bootstrap=True,
    criterion='entropy',
    random_state=42
)
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)
acc = accuracy_score(y_test, y_pred)
print(f"Overall Test Accuracy: {acc * 100:.1f}%")

Typical result on this split: about 94.2% test accuracy.

5. Confusion Matrix & Metrics

Example confusion matrix (171 test samples):

	Pred. Class 1	Pred. Class 2
Actual Class 1	58	6
Actual Class 2	4	103

• Diagonal = correct (58 + 103)
• Off-diagonal = errors (6 + 4)

Approximate metrics: precision ~94% both classes; recall ~91% / 96%; F1 ~92% / 95%. Top feature importances often include Worst Concave Points (~0.16) and Worst Area (~0.14).