Lesson 6: Pruning and distillation: fewer weights, smaller models

Module 11 · Lesson 6 · About 60 minutes · Needs: Module 6 (the base recipe); Lesson 5 of this module helps but is not required

The situation

Sol 434 at Aurora Station, the second day without the Earth link. The station's CPU servers run base for every crew request, beside the robots and greenhouse controllers. Lesson 5 stored each weight in fewer bits. Hana Aoki asks the next question: "Can we throw weights away? Or train a smaller model that behaves like base?"

You test both on one log header, "Log 4821, sol 212, the night shift.", and base's continuation of it. For every smaller model you ask: how much worse, how much faster, and how often it still says what base says.