Lesson 2: Finding circuits and features inside a model
31 min read·13 Oct 2026
Module 14 · Lesson 2 · About 75 minutes · Needs: Lesson 1 (hooks, exact head ablation, direct logit attribution); Module 2, Lesson 3 (heads on GPT-2); shared blocks 1 to 9 and 15 to 19
The situation
Sol 452 at Aurora Station. Lesson 1 caught a 100% model right for the wrong reason: every copy in the shared induction task sits 16 back, so "read 15 back" scored perfectly. Kofi Mensah: "Train a model that cannot cheat, show me its mechanism, then use the same tools on base and a real model."
The running example is held-out row 3584 of a new task with a different gap per row. At position 30 it reads token 10; every earlier 10 was followed by 25, so the answer is 25.