Lesson 2: Optimisers, schedules and the step-22 spike
28 min read·13 Oct 2026
Module 6 · Lesson 2 · About 55 minutes · Needs: Lesson 1 (the training loop, train, the proxy); Module 5, Lesson 5 (gradients)The situation
Sol 412 at Aurora Station. Kofi Mensah has base's log open at step 22: 4.235, between 3.112 and 3.150. "Was that a bad batch? Why AdamW, why 3e-3, why warm-up and cosine? And when a paper says a new optimiser is twice as fast, how do we check?"
This lesson opens the optimiser and schedule Lesson 1 fixed and runs Kofi's experiments. The running example is base's step 22: what the optimiser did at step 21, and why.