Lab: Efficient attention, KV cache and long-context probes, tested

Module 10 · Lab · About 60 minutes · Needs: lessons 1 to 4 of this module, and Module 8's lab

What you are building

Sol 431 at Aurora Station. Tariq Haddad's relay still splits long transmissions, and the four lessons answered him in notebook cells: sparse and banded attention (Lesson 1), linear attention (Lesson 2), tiled attention and the KV cache (Lesson 3), and a report card for long reads (Lesson 4).

Today those cells become three files in tinyformer-lab: attention_variants.py, kvcache.py and longctx.py. Course-guide/03 lists the first two for this module but gives Lesson 4's probes no home, so this lab chooses one: tinyformer/longctx.py. You pin the claims with nine tests, rerun the module's main table from the package, and tag v0.10.0.