Chapter 10
Interpretability: What a Representation Reveals, and What a Model Uses
Chapter: Interpretability, in Daniel Jurafsky and James H. Martin, Speech and Language Processing, 3rd edition draft, August 19, 2026. The source chapter is a partial draft.
Topics to learn
Page numbers refer to the linked chapter PDF.
- Contextual embeddings p. 1
- Word sense p. 2
- Word similarity, and why centering or rescaling can change cosine p. 4
- Readouts pp. 5-6
- Probes p. 5
- The logit lens p. 6
- Methods the partial chapter introduces p. 8
- Sparse autoencoders p. 8
- Causal interventions p. 8
- Induction heads and in-context learning p. 8
FAQ
Does a strong word-sense classifier prove the model uses dictionary senses?
No. It shows that the extracted vectors support that external classifier. The model may keep a useful distinction without using that sense inventory.
A probe is perfectly accurate. What else should you ask?
Whether the model's own output depends on that information. A readout can see a feature the rest of the computation ignores.
Does the logit lens have to name the model's final answer?
No. Later layers can change the hidden state. The lens applies the output map early. It is not the finished network.
In an induction pattern, does the current position attend to a future copy?
No. It attends backward to an earlier match. Information from that earlier position can then affect the next-token prediction.