Lab: Instruments, circuits and a state-space layer

Module 14 · Lab · About 60 minutes · Needs: lessons 1 to 4 of this module, and the Module 13 lab

What you are building

Sol 452 at Aurora Station. Kofi Mensah asked what base has learnt. Four lessons built the instruments to answer him, each in its own cells, and left three things for the lab to settle.

Three induction tasks: the shared TASKS["induction"] puts every copy 16 back, and Lesson 1 caught a model passing it with "read 15 back". Lesson 2's make_varied gives each row its own gap, and two layers reach 0.999 with real previous-token and induction heads. Lesson 3's gap_task levelled at 0.68. Lesson 4 added make_recall. The package keeps the shared task as the shortcut example, Lesson 2's recipe as the induction task, and Lesson 4's recall.

Today they become tinyformer/analysis.py and tinyformer/ssm.py, nine tests, one experiment, tag v0.14.0.