Lesson 1: Scaling laws, and what a small grid can say

Module 9 · Lesson 1 · About 45 minutes · Needs: Module 6 (the base recipe, the seed noise floor); Module 8, Lesson 4 (6ND and tokens per parameter)

The situation

Sol 426 at Aurora Station. Hana Aoki, the Compute Officer, holds the budget for GPU time hired on Earth, capped at $1,500 a run. She asks: "What would a model 1,000 times bigger than base buy us?"

You cannot train that model here, but you can train small ones, fit a curve and read it at the size you cannot afford. Kofi Mensah adds his rule: the answer comes with error bars, or it does not go to Hana. The running example is Hana's question and the ten-run grid that tries to answer it.