AI Courses for Kids in Marseille

Chapter 125 – Naive Bayes Independence Assumption

1. The Core Feature Independence Assumption

Naive Bayes is built on Bayes' Theorem. Its defining trait is the assumption that every input feature is independent of the others given the class label, so each feature contributes separately to the class decision.

2. Real-World Interdependence vs. the Naive Assumption

In real data, features usually interact (the "Butterfly Effect" idea: small changes can cascade). Naive Bayes still treats features as independent. That trade-off ignores weak interactions so joint probabilities become a simple product of individual conditionals—much faster to train and predict.

3. Why We Multiply Probabilities

Under independence:

P(X₁, X₂, …, Xₙ | Class) = P(X₁ | Class) × P(X₂ | Class) × … × P(Xₙ | Class)

Without the assumption you would need large multi-dimensional contingency tables. With it, the math stays simple and fast.

4. Spam Example: "Win" and "Prize"

In real language those words often appear together. Naive Bayes still treats them as separate events and multiplies:

P("Win", "Prize" | Spam) = P("Win" | Spam) × P("Prize" | Spam)

5. Python: GaussianNB on Breast Cancer

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import accuracy_score

cancer = load_breast_cancer()
X, y = cancer.data, cancer.target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42)

gnb = GaussianNB()
gnb.fit(X_train, y_train)
y_pred = gnb.predict(X_test)
print(accuracy_score(y_test, y_pred) * 100)  # ~93.71%

Defaults: priors=None (learned from class counts), var_smoothing=1e-09 for numerical stability.

6. Inspecting Internals

• class_prior_ — baseline class probabilities
• theta_ — feature means per class
• var_ — feature variances per class

Despite 30 correlated biological features, GaussianNB still reached about 93.71% test accuracy by modeling each feature as an independent Gaussian and multiplying the densities.

7. Summary

Aspect	Real world	Naive Bayes	Impact
Feature links	Interdependent	Independent	Ignores weak correlations
Joint probability	Heavy matrices	Product of singles	Fast math
Spam words	Correlated	Separate events	Simple product
Breast cancer (30 feats)	Correlated measures	Independent Gaussians	~93.71% accuracy