CourseTransformers: From Attention to Trained, Served and Interpreted Models · Module 7: Architecture Families and Design Choices · part 52 of 111
Part 52 · Module 7: Architecture Families and Design Choices

Lesson 3: The modern default stack, measured one switch at a time

24 min read·13 Oct 2026
Module 7 · Lesson 3 · About 60 minutes · Needs: Lesson 2 of this module (the 2017 preset, diff); Module 6, Lesson 3 (ablation methodology, verdict)

The situation

Sol 418 at Aurora Station. Lesson 2 showed that base is the 2017 transformer with five switches replaced. Kofi Mensah is not satisfied: "Five replacements, and we copied all of them. Which ones earn their place on our corpus? One change at a time, matched weights, seeds."

This lesson walks from the 2017 design to base one measured switch at a time, then varies the shape at a fixed weight count. The running example is the 2017 preset at base's size, followed until it becomes base.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.