Part 3: From raw model to useful assistant
The goal of this part is understanding the gap between a pretrained model and something a person can use.
Under construction
This arc is being written. The list below is what it will cover, and it may still change.
- Instruction tuning and preference trainingSupervised fine-tuning on demonstrations. Reward models. RLHF and the simpler methods that replaced it. What alignment training does to a model's behaviour and what it costs in capability. Why a base model and a chat model feel like different species.
- Fine-tuningFull fine-tuning, LoRA, and adapters. What fine-tuning is genuinely good at, which is style, format, and narrow tasks. What it is bad at, which is teaching new facts. A decision procedure for choosing between prompting, retrieval, and fine-tuning.
- PromptingWhy phrasing changes output, in terms of the mechanisms from Part 2. Few-shot examples, structured output, chain of thought and what it really does. The techniques that survive contact with real systems, and the ones that are folklore.
- EvaluationThe hardest unsolved practical problem in the field. Benchmarks and why they mislead. Building a test set for your own task. Model-graded evaluation, its failure modes, and how to check the grader. Regression testing when the output is text.