Lesson 3: Stable runs, and the settings that matter

Module 6 · Lesson 3 · About 50 minutes · Needs: Lesson 1 (the recipe, the proxy, the harness); Module 5, Lesson 2 (initialisation)

The situation

Sol 412 at Aurora Station, later the same day. base's recipe says learning rate 3e-3. Kofi Mensah asks who chose it. "What happens at ten times that? What warns you before the loss does? Would it hold on a bigger model?" Hana Aoki, who owns the compute, adds: "Spend CPU time only on the settings that matter."

This lesson answers on Lesson 1's proxy. The running example is the hot run: the proxy at learning rate 0.1, watched through one validation window, "... Bruno Hoffman works in th".