CourseTransformers: From Attention to Trained, Served and Interpreted Models · Module 4: Positional Information · part 32 of 111
Part 32 · Module 4: Positional Information

Lab: Every position method behind one switch

26 min read·13 Oct 2026
Module 4 · Lab · About 75 minutes · Needs: lessons 1 to 4 of this module, and Module 3's lab

What you are building

TINYFORMER-LAB AFTER MODULE 4 positions.py every method, one file config, attention, model small exact edits test_m04 12 claims m04_positions train 64, read 256 Module 3 files v0.3.0 git tag v0.4.0
The lab saves every position method into one file and wires it into Module 3's code (blue), holds it with twelve tests (green), runs one seeded length experiment (amber) and tags v0.4.0.

Sol 408 at Aurora Station. Each lesson in this module ended "the module lab saves this", and TinyFormer still refuses its own default: position="rope" raises an error. Kofi Mensah's question from Lesson 2 is also open: rotary was the most fragile method on one seed. Does that hold on the package's model?

Today you save every method into tinyformer/positions.py, wire the switch into ModelConfig and the model, pin each lesson's claim with a test, then run one train-short, read-long comparison with three seeds. You commit and tag v0.4.0.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside