NLP: Reading Guide
Natural language processing turns language into something a computer can analyze, predict or generate. The central questions fit together: what task are we solving, how do we represent its text, what does the model learn, and how do we tell whether its output helps?
Begin with the textbook path below. It builds the ideas with small examples before introducing larger systems. Use the research-paper collection when you want to follow a particular idea back to its source.
Popular textbooks
Chapter links, topic checklists, and FAQs. The Speech and Language Processing notes follow the August 19, 2026 draft.
Chapter links, topics, and FAQs for the August 19, 2026 draft. Every source chapter and appendix has an outline. Chapter 12 has no source PDF yet.
A first route: task → tokens → probabilities
Keep one question in mind as you move through the first three chapters: how does a system turn a piece of language into a useful prediction?
- Start with language models and useful systems. Understand what next-token prediction supplies and where the surrounding task, data and evaluation enter.
- Make the input explicit. Follow words, characters and subwords into token sequences; work through the representation before reasoning about a model.
- Build a language model from counts. Calculate conditional probabilities, score a sequence and see why unseen combinations create a modeling problem.
The introduction is a conceptual starting point. For the probability calculations, revisit probability foundations and entropy and log-loss. For experiments, use train, validation and test splits to separate learning from evaluation.
LLM evaluation datasets & benchmarks
Connect the foundations to another question
- How are pretrained and fine-tuned LLMs benchmarked? Follow the LLM evaluation datasets & benchmarks series from Bits-Per-Byte perplexity to verifiable code execution and preference judges.
- Where did these ideas come from? Explore the existing NLP research-paper collection, from information theory through neural language models and transformers.
- How can text become features for a simple model? Work through bag of words and TF-IDF.
- How do representations help find information? Continue to the search and retrieval guide, then study its evaluation path.
For each chapter, try the small example yourself before reading the result. Then explain which quantities came from data, which were design choices, and which would need to be measured on unseen examples. That habit carries from count-based models to much larger language systems.