Appendix B
Naive Bayes Classification
Appendix: Naive Bayes Classification, in Daniel Jurafsky and James H. Martin, Speech and Language Processing, 3rd edition draft, August 19, 2026.
Topics to learn
Page numbers refer to the linked chapter PDF.
- The classifier p. 2
- The naive independence assumption p. 2
- Training from labeled documents p. 5
- A worked example p. 6
- Text uses p. 7
- Sentiment optimizations p. 7
- Other classification tasks p. 9
- Naive Bayes as a language model p. 10
- Evaluation p. 11
- Precision, recall, and F-measure p. 11
- More than two classes p. 13
- Test sets and cross-validation p. 13
- Significance testing and the paired bootstrap p. 14
- Harms that an average score can hide p. 18
FAQ
What is naive about naive Bayes?
It treats features as independent given the class. Words in a document are not independent. The assumption is a modeling choice that makes the estimate a product of feature likelihoods.
Does a high class probability mean the decision threshold is the only thing left?
No. The probability comes from the model and the smoothing. Precision and recall still depend on the threshold and on which errors matter.
How can naive Bayes be a language model?
A class-conditional word model assigns probability to a document given a class. Comparing classes is classification. Generating or scoring text under one class is the language-model view.
Why test significance on the same items?
A paired test asks whether a score difference is large relative to resampling those items. It does not replace a new test set, and it does not measure harm to a subgroup.