CourseTransformers: From Attention to Trained, Served and Interpreted Models · Module 3: The Transformer Block · part 25 of 111
Part 25 · Module 3: The Transformer Block

Lab: One config, one block, every switch

27 min read·13 Oct 2026
Module 3 · Lab · About 75 minutes · Needs: lessons 1 to 4 of this module, and Module 2's lab

What you are building

TINYFORMER-LAB AFTER MODULE 3 config.py every switch layers.py FFN and norms model.py Block, TinyFormer attention.py Module 2 test_m03_block 12 claims m03_switches 3 arms x 3 seeds git tag v0.3.0
The lab adds three package files (blue) on top of Module 2's attention (grey), twelve tests that hold them to the lessons (green), one seeded comparison (amber) and the v0.3.0 tag.

Sol 407 at Aurora Station. Each of the four lessons grew its own copy of ModelConfig, and the copies disagree: Lesson 3's block takes no mask and brings its own attention class. Kofi Mensah asks: "Which one is the model?"

Today there is one answer: one ModelConfig with every switch, one Block that reads them all and takes a mask, built on Module 2's tinyformer/attention.py, and TinyFormer on top. Twelve tests pin each switch to what its lesson showed. Then one seeded comparison of two switches, a commit and the tag v0.3.0.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside