Scratch Coding Classes for Kids in Parnu

Chapter 124 – Naive Bayes Explained

1. Probabilistic Classification

Unlike models that only draw hard boundaries, Naive Bayes estimates a probability (0 to 1) that a sample belongs to each class, then picks the class with the highest posterior probability.

2. Why "Naive"?

It assumes features are independent given the class. Real data often has correlations, but independence turns joint probabilities into a product of simple conditionals—much faster to compute.

3. Bayes' Theorem

P(A | B) = [P(B | A) · P(A)] / P(B)

• Posterior P(A|B): probability of class A given features B (what we predict)
• Prior P(A): base rate of class A in training
• Likelihood P(B|A): how often features B appear in class A
• Evidence P(B): overall rate of features B

4. Spam Example

For words like "Winner" or "Free": use P(words | spam), P(spam), and P(words) to get P(spam | words). Compare to P(not spam | words) and assign the higher posterior.

5. Strengths and Limits

Strengths	Limits
Fast on large data	Independence often unrealistic
Works with small samples	Weak when features are highly correlated
Strong on text / categorical data	Best when feature dependence is mild

Common uses: text classification, spam filtering, sentiment analysis.

6. Term Summary

Term	Role
Probabilistic model	Outputs class probabilities 0–1
Naive assumption	Features independent given class
Prior / Likelihood / Posterior	Base rate / feature given class / class given features