Naive Bayes
In one sentence Naive Bayes classifies by multiplying together how strongly each feature hints at each class, assuming the features do not influence each other.
Updated
Naive Bayes is a classifier that combines the evidence from each feature independently, using probability rules, to pick the most likely class.
It reasons the way you skim a suspicious email. "Lottery" — that word alone raises your suspicion. "Winner!!!" — more suspicion. "Click here immediately" — more still. You never analyse how the words interact; each clue independently nudges your belief, and the nudges pile up to a verdict. Naive Bayes does exactly this with counted probabilities: how often does "lottery" appear in spam versus real mail?
The "Bayes" part is Bayes' theorem, the rule for updating a belief when evidence arrives. The "naive" part is the big simplifying assumption: features are treated as independent of each other within a class. For language this is false — "Sri" and "Lanka" travel together — yet the classifier works far better than the broken assumption suggests, because classification only needs the ranking of classes to be right, not the probabilities themselves.
Its virtues are speed and appetite: it trains in one pass of counting, needs little data, handles thousands of features, and updates cheaply as new examples arrive. It powered a generation of spam filters and is still the fastest respectable baseline for text classification. Expect logistic-regression or fine-tuned transformers to beat it — after costing considerably more.
Where to go next
- Full lesson: Classification
- Related terms: classification, logistic-regression, baseline, supervised-learning