Chapter 4
Logistic Regression for Text: From Word Counts to a Decision
Chapter: Logistic Regression and Text Classification, in Daniel Jurafsky and James H. Martin, Speech and Language Processing, 3rd edition draft, August 19, 2026.
Topics to learn
Page numbers refer to the linked chapter PDF.
- Classification setup pp. 2-8
- Features, labels, and held-out evaluation p. 2
- The sigmoid p. 3
- Sentiment and other text features p. 6
- Scoring many examples at once p. 8
- Learning pp. 9-16
- Cross-entropy loss p. 10
- Gradient descent p. 12
- The logistic gradient p. 14
- Stochastic and mini-batch updates pp. 14-16
- A worked update in the chapter p. 15
- More than two classes pp. 17-20
- Softmax p. 18
- Class-specific features p. 19
- Learning the multinomial model p. 20
- Evaluation and comparison pp. 21-27
- Precision, recall, and F-measure p. 21
- Multiclass evaluation p. 23
- Test sets and cross-validation p. 24
- Statistical significance and the paired bootstrap p. 25
- Use and interpretation pp. 28-32
- Harms in classification p. 28
- Reading feature weights p. 30
- Regularization p. 30
- Where the gradient equation comes from p. 32
FAQ
Why does the binary logistic gradient simplify to (p - y) times the feature?
The loss derivative with respect to the probability, times the sigmoid derivative, cancels to p - y. The feature then scales how much that weight contributed to the logit.
Does changing the decision threshold change cross-entropy?
No. The probabilities stay fixed, so the probability loss stays fixed. The threshold changes labels, and therefore precision, recall, and F1.
What is a paired bootstrap checking?
Whether a score difference on the same test items is large relative to differences you get by resampling those items. It is not a substitute for a fresh test set.
Can a high accuracy still be a harmful classifier?
Yes. Errors, labels, and features can fall unevenly across groups. The chapter treats that as a separate evaluation question from the average score.