Online Robotics Classes for Kids in Zagreb

Chapter 108 – Build a Decision Tree Model with Python and Scikit-Learn

1. End-to-End Project Workflow

This project loads the Iris dataset, applies pre-pruning, trains a DecisionTreeClassifier, visualizes the tree, and evaluates with Accuracy, Confusion Matrix, Precision, Recall, and F1-Score.

• Data ingestion (Iris features and labels)
• Stratified 70/30 train–test split
• Pre-pruned model: max_depth=3, criterion='entropy'
• Tree visualization and traversal
• Evaluation: ~93.3% accuracy, 3×3 confusion matrix, class metrics

2. Dataset and Stratified Split

Iris features: petal/sepal length and width. Classes: Setosa, Versicolor, Virginica.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, stratify=y, random_state=42
)

stratify=y keeps the same class proportions in train and test (e.g. 35 per class in train, 15 per class in test), avoiding imbalance skew.

3. Model Construction and Pre-Pruning

from sklearn.tree import DecisionTreeClassifier

clf = DecisionTreeClassifier(
    max_depth=3,
    criterion='entropy',
    random_state=42
)
clf.fit(X_train, y_train)

• max_depth=3 — limits depth to reduce overfitting
• criterion='entropy' — splits by Information Gain
• random_state=42 — reproducible results

4. Interpreting the Tree

Root: Petal Length ≤ 2.45? (Entropy ≈ 1.585, 105 samples).

• True → pure Setosa leaf (35 samples, Entropy 0)
• False → Petal Width ≤ 1.75 on remaining 70 samples, then further splits on petal/sepal length into four leaves (e.g. pure Versicolor, pure Virginica, and a mixed leaf that votes majority)

5–6. Evaluation and Confusion Matrix

from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

y_pred = clf.predict(X_test)
print(accuracy_score(y_test, y_pred))  # ~0.933 (93.3%)
print(confusion_matrix(y_test, y_pred))

Typical 3×3 pattern on 45 test samples: Setosa 15/15 correct; Virginica 15/15 correct; Versicolor 12/15 correct with 3 misclassified as Virginica. The entire ~6.7% error comes from Versicolor ↔ Virginica confusion.

7. Classification Report Metrics

Class	Precision	Recall	F1-Score	Support
Setosa	1.00	1.00	1.00	15
Versicolor	1.00	0.80	0.89	15
Virginica	0.83	1.00	0.91	15
Overall	—	—	0.93	45

• Precision — of predicted positives, how many were correct
• Recall — of actual positives, how many were found
• F1 — harmonic mean of precision and recall
• Support — true sample count per class

8. Complete Script Outline

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
import matplotlib.pyplot as plt

iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, stratify=y, random_state=42
)
clf = DecisionTreeClassifier(max_depth=3, criterion='entropy', random_state=42)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print(accuracy_score(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred, target_names=iris.target_names))
plt.figure(figsize=(12, 8))
plot_tree(clf, feature_names=iris.feature_names, class_names=iris.target_names, filled=True)
plt.show()