Python Coding Classes for Children in Limassol

Chapter 113 – Random Forest Hyperparameter Tuning

1. Parameters vs Hyperparameters

• Parameters — learned during training (e.g. split thresholds, weights).
• Hyperparameters — set by you before training (tree count, depth, min samples, etc.).

Tuning aims for good performance on unseen data and a sensible bias–variance balance (avoid under- and overfitting).

2. Main Random Forest Hyperparameters

Hyperparameter	Role
n_estimators	Number of trees (often start 100–200; more trees stabilize but cost more time)
max_depth	Max tree depth (shallow → underfit; unlimited → overfit; often try 5–20)
min_samples_split	Min samples needed to split a node (higher → smoother trees)
min_samples_leaf	Min samples in a leaf (higher → less noise memorization)
max_features	Max features considered per split (boosts diversity across trees)

3. Confusion Matrix Basics

• TP — predicted positive, correct
• TN — predicted negative, correct
• FP (Type I) — predicted positive, wrong
• FN (Type II) — predicted negative, wrong

These feed Precision, Recall, and F1-Score.

4. GridSearchCV

Define a param_grid, then run k-fold CV (e.g. cv=5) over every combination; pick the set with the best average score. Large grids grow exponentially — keep candidates focused to control runtime.

5. Python Example (Breast Cancer)

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report

cancer = load_breast_cancer()
X, y = cancer.data, cancer.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, stratify=y, random_state=42
)

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 5, 10, 20],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4]
}

rf_base = RandomForestClassifier(random_state=42, n_jobs=-1)
grid_search = GridSearchCV(
    estimator=rf_base,
    param_grid=param_grid,
    cv=5,
    scoring='accuracy',
    n_jobs=-1
)
grid_search.fit(X_train, y_train)

best_model = grid_search.best_estimator_
y_pred = best_model.predict(X_test)
print(grid_search.best_params_)
print(f"Test Accuracy: {accuracy_score(y_test, y_pred) * 100:.2f}%")
print(classification_report(y_test, y_pred, target_names=cancer.target_names))

6. Example Outcome

A typical best set: n_estimators=100, max_depth=None, min_samples_split=2, min_samples_leaf=1 → ~93.6% test accuracy. More trees (e.g. 200) did not improve accuracy here, so the search preferred 100 for efficiency.