Python Coding Classes for Children in Antwerp

1. Goal of Splitting

Trees grow by asking feature questions that move data from mixed (high entropy) subsets toward purer (low entropy) leaves. Metrics such as Information Gain and Gini Index decide which feature to split on at each node.

2. Structure Recap

• Root — full dataset, first split
• Decision nodes — further feature tests
• Leaves — final class or value
• Branches — outcomes of each test

3. Entropy, Information Gain, Gini

• Entropy — uncertainty/noise; 0 = pure; high when classes are mixed (e.g. 50–50)
• Information Gain — drop in entropy after a split; the tree prefers the feature with the largest gain
• Gini Index — class mixing; lower Gini means purer subsets

4. Pure vs Impure

Impure example: 3 of class A and 3 of class B → max confusion. Pure example: all samples one class → entropy ≈ 0.

5. Overfitting

Unlimited growth can memorize training noise. Classic sign: very high train accuracy (e.g. > 90%) but poor test accuracy (e.g. 20–30%).

6. Pruning

• Pre-pruning — stop early (max depth, min samples, min gain)
• Post-pruning — grow fully, then cut weak branches

Both aim for generalization: learn domain patterns, not rote training details.