Part 10 · Module 1: The Road to Attention

Lab: Start tinyformer-lab with one honest race

16 min read·13 Oct 2026

Module 1 · Lab · About 50 minutes · Needs: lessons 1, 2 and 3

What you are building

By the end of the lab the package (blue) feeds one race script, tests (green) guard the data, the script writes nine measured runs to a record (amber), and Git tags the whole state v0.1.0.

Lessons 2 and 3 raced models on one seed each, and Kofi Mensah's rule at Aurora Station is plain: no comparison without a seed noise floor. On sol 400 you start tinyformer-lab, put the shared data into its package with a test, and race three models of about 105,000 parameters on the shared copy task: a plain GRU, a GRU with attention over its own past states, and a one-layer transformer encoder with a causal mask. Each runs on three seeds. One script writes every result to a JSON run record in runs/, and you tag the repository v0.1.0. Every number is labelled measured here, cited or arithmetic.

Step 1: Start the repository

Four folders with one job each: code you import, claims you test, scripts that make tables, and results, which never live inside the package.

A repository has a fixed shape so anyone can find things. The package tinyformer/ holds code you import, tests/ holds checks, experiments/ holds scripts that make tables, and runs/ holds their results. Start a fresh Colab notebook on the CPU runtime and run this cell first.

python

import os, pathlib
if pathlib.Path.cwd().name != "tinyformer-lab":          # safe to rerun: never nests a second copy
    os.makedirs("tinyformer-lab", exist_ok=True)
    os.chdir("tinyformer-lab")                           # in Colab: /content/tinyformer-lab
for folder in ["tinyformer", "tests", "experiments", "runs"]:
    pathlib.Path(folder).mkdir(exist_ok=True)

FILES = {
    "pyproject.toml": '[build-system]\nrequires = ["setuptools>=64"]\nbuild-backend = "setuptools.build_meta"\n\n'
                      '[project]\nname = "tinyformer"\nversion = "0.1.0"\nrequires-python = ">=3.10"\n'
                      'dependencies = ["torch"]\n\n[tool.setuptools]\npackages = ["tinyformer"]\n',
    ".gitignore": "__pycache__/\n*.pyc\n*.egg-info/\n.ipynb_checkpoints/\n",
    "README.md": "# tinyformer-lab\n\nA small transformer in which every design choice is a switch you can measure.\n"
                 "Every number in this repository is labelled measured here, cited or arithmetic.\n",
    "tinyformer/__init__.py": '"""tinyformer-lab: the package. Module 1 adds data.py."""\n__version__ = "0.1.0"\n',
}
for path, text in FILES.items():
    pathlib.Path(path).write_text(text)
print(os.getcwd().split("/")[-1], sorted(str(p) for p in pathlib.Path(".").rglob("*") if ".git" not in str(p)))

Code walkthrough

The if moves into tinyformer-lab only once, so running the cell twice does not make tinyformer-lab/tinyformer-lab.
The loop makes the four folders; exist_ok=True skips any that exist.
pyproject.toml tells pip the package's name, version and only dependency, PyTorch. The [build-system] table names setuptools, the tool that installs it.
.gitignore lists files Git should never store.
tinyformer/__init__.py makes the folder a package and records __version__, which you raise with every module's tag.

You should see tinyformer-lab and eight sorted paths: ['.gitignore', 'README.md', 'experiments', 'pyproject.toml', 'runs', 'tests', 'tinyformer', 'tinyformer/__init__.py'].

Step 2: Put the shared data in the package

Shared blocks 1 and 9 become one importable file, unchanged except that their print lines (grey) are dropped so importing prints nothing.

Lessons 2 and 3 pasted shared blocks 1 and 9 from course-guide/06 into every notebook. From now on they live in one file and you import them. The code is block 1 and block 9 exactly, without their print lines and the bare set_seed() call, so importing the package prints nothing and changes no random state. The rest of 06 (the corpus, tokeniser and batches) is appended by the first lab that trains on text.

python

%%writefile tinyformer/data.py
"""Shared data for tinyformer-lab, from course-guide/06. Module 1: seeds (block 1) and synthetic tasks (block 9)."""
import random
from typing import NamedTuple

import torch

COURSE_DATE = "2026-10-09"   # versions, model names and prices checked on this day
SEED = 11                    # every generator in the course starts from this seed
SNAPSHOT_SOL = 400           # the corpus is the station record up to sol 399

def set_seed(seed=SEED):
    """Seed Python and PyTorch so a cell gives the same numbers every run."""
    random.seed(seed)
    torch.manual_seed(seed)

Code walkthrough

%%writefile saves the cell's text as the named file instead of running it.
The constants and set_seed are block 1 of course-guide/06, unchanged.

python

%%writefile -a tinyformer/data.py
class TaskData(NamedTuple):
    name: str
    vocab_size: int
    inputs: torch.Tensor    # (n, T) token ids
    targets: torch.Tensor   # (n, T) the next token at every position
    mask: torch.Tensor      # (n, T) 1 where the answer is scored, 0 elsewhere

def _shift(name, vocab, seq, scored_from):
    inputs, targets = seq[:, :-1], seq[:, 1:]
    mask = torch.zeros_like(inputs)
    mask[:, scored_from:] = 1
    return TaskData(name, vocab, inputs, targets, mask)

def make_task(name, n=4096, length=16, vocab=32, seed=SEED):
    """copy, reverse or induction. Token 0 is the separator; content tokens are 1 to vocab - 1."""
    g = torch.Generator().manual_seed(seed)
    body = torch.randint(1, vocab, (n, length), generator=g)
    sep = torch.zeros(n, 1, dtype=torch.long)
    if name == "copy":
        seq = torch.cat([body, sep, body], dim=1)
    elif name == "reverse":
        seq = torch.cat([body, sep, body.flip(1)], dim=1)
    elif name == "induction":                       # random tokens, then the same random tokens again
        seq = torch.cat([body, body], dim=1)
    return _shift(name, vocab, seq, scored_from=length)
TASKS = {name: make_task(name) for name in ["copy", "reverse", "induction"]}

Code walkthrough

-a appends to the file instead of replacing it, so data.py now holds block 1 followed by block 9, unchanged apart from its print loop.
TASKS is built on import, in milliseconds.

You should see Writing tinyformer/data.py, then Appending to tinyformer/data.py.

Step 3: Test the data

The data tests sit between any edit and any experiment: green lets the race run, red (a failure) stops it before a result is built on changed data.

A test is a claim the repository checks every time it changes. These four are the ones the race depends on: the copy task's shape, answers 16 positions back, a seed that fixes the data, and a random unscored half. If a later module breaks one, pytest says so before any experiment runs on bad data.

python

%%writefile tests/test_m01_data.py
import torch
from tinyformer.data import SEED, TASKS, make_task

def test_copy_shape_and_mask():
    copy = TASKS["copy"]
    assert tuple(copy.inputs.shape) == (4096, 32)
    assert copy.mask[:, :16].sum() == 0 and bool((copy.mask[:, 16:] == 1).all())

def test_copy_answer_is_sixteen_back():
    copy = TASKS["copy"]
    assert torch.equal(copy.targets[:, 16:], copy.inputs[:, :16])

def test_seed_fixes_the_data():
    assert torch.equal(make_task("copy").inputs, TASKS["copy"].inputs)
    assert not torch.equal(make_task("copy", seed=SEED + 1).inputs, TASKS["copy"].inputs)

def test_unscored_half_is_random_tokens():
    body = TASKS["copy"].inputs[:, :16]
    assert body.min() == 1 and body.max() == 31          # 31 content tokens, separator 0 never inside
    assert torch.bincount(body.flatten())[1:].min() > 1800   # every token appears: no token is rare

python

%pip install -q -e . pytest
!python -m pytest -q tests

Code walkthrough

Each test_ function is one claim; pytest finds and runs every function whose name starts with test_.
torch.equal(copy.targets[:, 16:], copy.inputs[:, :16]) checks all 4,096 rows: every scored target equals the input 16 positions earlier.
The last test counts each content token in the unscored half. 65,536 tokens over 31 values average about 2,114 each (arithmetic), so 1,800 is a loose floor.
%pip install -e . installs the package in editable mode: scripts can import tinyformer, and edits to its files take effect without reinstalling.

You should see 4 passed and a time under a second.

Step 4: Write the three racers

The second racer runs a GRU (blue states) and lets each position attend (amber) over its own state and every earlier one; later states are masked (grey).

The race script lives in experiments/: it uses tinyformer but is not part of it. The plain GRU is Lesson 2's racer. The second racer is new: Lesson 3's multiplicative attention, but the GRU attends over its own earlier states in a single stream, with a mask so position t sees only positions up to t. Its width, 104, brings it to 104,760 parameters against the GRU's 107,296.

python

%%writefile experiments/m01_rnn_vs_attention.py
"""Module 1 race: GRU, GRU with attention, one-layer transformer encoder, on the shared copy task."""
import json, platform, sys, time
import torch
import torch.nn as nn
import torch.nn.functional as F
from tinyformer.data import TASKS, set_seed

class GRUModel(nn.Module):                         # Lesson 2's racer, unchanged
    def __init__(self, vocab=32, d=128):
        super().__init__()
        self.embed, self.gru, self.head = nn.Embedding(vocab, d), nn.GRU(d, d, batch_first=True), nn.Linear(d, vocab)

    def forward(self, ids):
        return self.head(self.gru(self.embed(ids))[0])

class GRUAttention(nn.Module):
    """A GRU whose every position attends over its own states so far (Luong's 'general' score)."""
    def __init__(self, vocab=32, d=104):
        super().__init__()
        self.embed, self.gru = nn.Embedding(vocab, d), nn.GRU(d, d, batch_first=True)
        self.Wa, self.combine, self.head = nn.Linear(d, d, bias=False), nn.Linear(2 * d, d), nn.Linear(d, vocab)

    def forward(self, ids):
        S, _ = self.gru(self.embed(ids))                                  # (B, T, d): every state
        T = ids.shape[1]
        future = torch.triu(torch.ones(T, T, dtype=torch.bool), diagonal=1)
        a = (S @ self.Wa(S).transpose(1, 2)).masked_fill(future, float("-inf")).softmax(-1)
        return self.head(torch.tanh(self.combine(torch.cat([S, a @ S], -1))))

Code walkthrough

GRUAttention runs the GRU over the whole row, then scores every state against every state with one matrix product, S @ Wa(S)ᵀ, shape (B, 32, 32).
future is True above the diagonal: the pairs where a position would look at a later one. masked_fill sets those scores to minus infinity, so softmax gives them weight 0.
a @ S is each position's context, a blend of its own and earlier states; combine mixes it with the state, as Luong did in Lesson 3.

The three racers are matched in size within 5%, and the two with attention (amber) are the smaller ones, so a win cannot come from extra parameters.

The third racer is the smallest transformer that can solve copy: one encoder layer from PyTorch, learned positions and a causal mask. Lesson 2 used two layers; one is enough here, because the answer sits a fixed 16 positions back and learned positions can point there directly. At width 88 it has 102,552 parameters, the fewest of the three.

python

%%writefile -a experiments/m01_rnn_vs_attention.py

class Encoder1(nn.Module):
    """One transformer encoder layer with learned positions and a causal mask."""
    def __init__(self, vocab=32, d=88, heads=4, context=32):
        super().__init__()
        self.embed, self.pos = nn.Embedding(vocab, d), nn.Embedding(context, d)
        self.layer = nn.TransformerEncoderLayer(d, heads, 4 * d, dropout=0.0, batch_first=True, norm_first=True)
        self.head = nn.Linear(d, vocab)

    def forward(self, ids):
        T = ids.shape[1]
        x = self.embed(ids) + self.pos(torch.arange(T))
        causal = nn.Transformer.generate_square_subsequent_mask(T)        # pass the mask AND the hint
        return self.head(self.layer(x, src_mask=causal, is_causal=True))

RACERS = {"gru": GRUModel, "gru_attention": GRUAttention, "encoder_1layer": Encoder1}
count = lambda m: sum(p.numel() for p in m.parameters())

Code walkthrough

Encoder1 is Lesson 2's AttentionModel with one layer, called directly.
src_mask=causal, is_causal=True passes both the mask and the flag that says it is causal. This module's incident file shows why both.
RACERS maps a name to each class, so the run record and the table use the same three names.

Step 5: Write the race and the run record

Nine runs: each racer on seeds 11, 12 and 13, and within a column every racer sees the same starting seed and the same batches.

A fair race changes one thing at a time. All three racers get the same 1,000 steps of 64 rows, the same optimiser and the same batches for a given seed; only the architecture differs. The seed sets both the starting weights and the batch order, and three seeds give three runs per racer. The record also logs accuracy on the unscored random half: it must stay near chance, 1 in 31, or a model is seeing answers it should not.

python

%%writefile -a experiments/m01_rnn_vs_attention.py

N_VALID, STEPS, SEEDS = 512, 1000, [11, 12, 13]       # last 512 rows held out; three seeds per racer

def scored_loss(model, x, y, mask):
    losses = F.cross_entropy(model(x).flatten(0, 1), y.flatten(), reduction="none")
    return (losses * mask.flatten()).sum() / mask.sum()

@torch.no_grad()
def accuracy(model, task):
    model.eval()
    x, y, m = (t[-N_VALID:] for t in (task.inputs, task.targets, task.mask))
    right = (model(x).argmax(-1) == y).float()
    model.train()
    return (right * m).sum().item() / m.sum().item(), (right * (1 - m)).sum().item() / (1 - m).sum().item()

def race(model, task, seed, steps=STEPS, every=50, lr=3e-3):
    set_seed(seed); g = torch.Generator().manual_seed(seed)
    opt, curve, t0 = torch.optim.AdamW(model.parameters(), lr=lr), [], time.perf_counter()
    for step in range(1, steps + 1):
        i = torch.randint(0, len(task.inputs) - N_VALID, (64,), generator=g)
        loss = scored_loss(model, task.inputs[i], task.targets[i], task.mask[i])
        opt.zero_grad(); loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0); opt.step()
        if step % every == 0:
            curve.append((step, round(accuracy(model, task)[0], 4)))
    scored, unscored = accuracy(model, task)
    first = next((s for s, a in curve if a >= 0.95), None)
    return {"seed": seed, "accuracy": scored, "unscored_accuracy": unscored, "first_step_0.95": first,
            "seconds": round(time.perf_counter() - t0, 1), "curve": curve}

Code walkthrough

scored_loss and the loop are Lesson 2's; new are the seed argument and the returned dictionary.
accuracy now returns two numbers: on the scored half (mask 1) and on the unscored half (mask 0).
first is the first logged step at which held-out accuracy reached 0.95, or None if it never did: a measure of how fast each racer learns, not only where it ends.

The run record keeps the measured numbers (amber) next to the environment that produced them (blue), so any number in a report can be traced to one file.

The run record makes the result checkable later. Beside the numbers it stores what produced them: versions, thread count, machine, date and settings. Kofi asks "which run?" and gets one file.

python

%%writefile -a experiments/m01_rnn_vs_attention.py

if __name__ == "__main__":
    task, runs = TASKS["copy"], []
    for name, make in RACERS.items():
        for seed in SEEDS:
            set_seed(seed); model = make()
            runs.append({"racer": name, "parameters": count(model), **race(model, task, seed)})
            r = runs[-1]
            print(f"{name:15} seed {seed}  acc {r['accuracy']:.4f}  unscored {r['unscored_accuracy']:.3f}"
                  f"  first>=0.95 {str(r['first_step_0.95']):>5}  {r['seconds']:5.1f} s", flush=True)
    record = {"experiment": "m01_rnn_vs_attention", "task": "copy, length 16", "label": "measured here",
              "date": time.strftime("%Y-%m-%d"), "python": platform.python_version(), "torch": torch.__version__,
              "threads": torch.get_num_threads(), "machine": platform.platform(),
              "settings": {"steps": STEPS, "batch": 64, "lr": 3e-3, "held_out": N_VALID, "seeds": SEEDS}, "runs": runs}
    with open("runs/m01_rnn_vs_attention.json", "w") as f:
        json.dump(record, f, indent=1)
    print("wrote runs/m01_rnn_vs_attention.json")

Code walkthrough

if __name__ == "__main__": runs the race only when the file runs as a script, so a notebook can import race safely.
set_seed(seed) before make() fixes the starting weights; race reseeds for the batches.
**race(...) merges the race's dictionary into the run's entry.
record is the run record. "label": "measured here" says what kind of numbers these are.

Step 6: Run the comparison

Each dot is one seed: the GRU's three runs (blue) spread from 0.568 to 0.589, while both attention racers (amber) sit at 0.997 to 1.000.

Run the script as Kofi would: in a fresh Python process, so nothing left in the notebook can affect it.

python

!python experiments/m01_rnn_vs_attention.py

Code walkthrough

! hands the line to the shell. The script imports tinyformer because Step 3 installed it.

You should see nine lines and the file name. On the author's Mac (Apple silicon, 5 threads, torch 2.14.1; measured here, 10 October 2026):

text

gru             seed 11  acc 0.5681  unscored 0.033  first>=0.95  None    5.6 s
gru             seed 12  acc 0.5886  unscored 0.032  first>=0.95  None    5.6 s
gru             seed 13  acc 0.5750  unscored 0.029  first>=0.95  None    5.6 s
gru_attention   seed 11  acc 0.9982  unscored 0.029  first>=0.95    50    5.9 s
gru_attention   seed 12  acc 0.9974  unscored 0.031  first>=0.95    50    5.9 s
gru_attention   seed 13  acc 0.9985  unscored 0.030  first>=0.95    50    5.9 s
encoder_1layer  seed 11  acc 1.0000  unscored 0.031  first>=0.95    50    3.3 s
encoder_1layer  seed 12  acc 1.0000  unscored 0.029  first>=0.95    50    3.3 s
encoder_1layer  seed 13  acc 1.0000  unscored 0.030  first>=0.95    50    3.4 s
wrote runs/m01_rnn_vs_attention.json

The GRU on seed 11 scores 0.5681, Lesson 2's 0.568 to the digits shown: the package reproduces the lesson. The whole script took 46 to 50 seconds on the Mac over two runs (measured here). A free Colab CPU has fewer, slower cores; allow two to four times longer.

Step 7: Read the result against the noise floor

The GRU's seed range (blue) is 0.0205 wide and the GRU with attention (amber) clears it by 0.4208, a gap the noise floor cannot explain (green, passed).

Three seeds of the same racer differ only by chance, so their spread is the noise floor: the smallest difference between racers you can believe. Kofi's rule makes it concrete. A racer beats another only if its worst seed beats the other's best seed, so the two ranges do not overlap.

python

import json
record = json.load(open("runs/m01_rnn_vs_attention.json"))
by = {}
for r in record["runs"]:
    by.setdefault(r["racer"], []).append(r["accuracy"])

print(f"{'racer':15} {'mean':>7} {'min':>7} {'max':>7} {'spread':>7}   (measured here, 3 seeds)")
for name, accs in by.items():
    print(f"{name:15} {sum(accs) / 3:7.4f} {min(accs):7.4f} {max(accs):7.4f} {max(accs) - min(accs):7.4f}")

def beats(a, b):
    """a beats b only if a's worst seed is above b's best seed."""
    return min(by[a]) > max(by[b])

for a, b in [("gru_attention", "gru"), ("encoder_1layer", "gru_attention")]:
    gap = sum(by[a]) / 3 - sum(by[b]) / 3
    print(f"{a} vs {b}: mean gap {gap:.4f}, ranges overlap: {not beats(a, b)}")
print("chance on a scored token:", round(1 / 31, 4), "(arithmetic)")

Code walkthrough

by groups the nine accuracies by racer; beats is Kofi's rule written down; 1 / 31 is chance over 31 content tokens.

You should see the GRU at mean 0.5772 with a spread of 0.0205, the GRU with attention at 0.9980 (spread 0.0011) and the encoder at 1.0000 (spread 0). Attention added to the GRU lifts it by 0.4208, about twenty times the GRU's noise floor (arithmetic: 0.4208 / 0.0205). The encoder's lead over the GRU with attention is 0.0020: its ranges do not overlap, but the gap is about 16 of the 8,192 scored answers per run (arithmetic). Both attention racers passed 0.95 by the first check at step 50, so neither learns faster on this measure. The difference that does clear the noise is cost: 3.3 seconds per run for the encoder against 5.9 for the GRU with attention (measured here, Mac), because the encoder has no 32-step loop.

Break it: race without a seed

With the seed the GRU scores 0.5681 every time (green); without it the same code gave 0.5734 then 0.5649 (red), and no result could be checked.

Take the seed away and the record stops being a record. Run the GRU twice as before, with nothing seeded.

python

import torch
from experiments.m01_rnn_vs_attention import GRUModel, scored_loss, accuracy, N_VALID
from tinyformer.data import TASKS
copy = TASKS["copy"]

def unseeded_race(steps=1000):
    model = GRUModel()                                   # weights from whatever random state is left
    opt = torch.optim.AdamW(model.parameters(), lr=3e-3)
    for _ in range(steps):
        i = torch.randint(0, len(copy.inputs) - N_VALID, (64,))   # batches from the global generator
        loss = scored_loss(model, copy.inputs[i], copy.targets[i], copy.mask[i])
        opt.zero_grad(); loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0); opt.step()
    return accuracy(model, copy)[0]

print("same code, run twice:", [round(unseeded_race(), 4) for _ in range(2)])

Code walkthrough

from experiments.m01_rnn_vs_attention import ... reuses the script's pieces. The __main__ guard stops the import from starting the race.
unseeded_race is the race with both seeds removed: the weights and the batches come from wherever the random state happens to be.

On the author's Mac it printed [0.5734, 0.5649] (measured here); yours will differ, from each other and from these. Neither can be checked against Lesson 2's 0.568, because nothing fixes what produced them. Without a seed, a 0.02 gap between two GRU variants could be design or luck, and nobody could tell.

Step 8: Commit and tag v0.1.0

Eight files go into one commit tagged v0.1.0; the push to GitHub (grey, dashed) is yours to run, because Colab's disk does not survive the runtime.

Colab wipes its disk when the runtime stops, so the repository is not safe until pushed. First record this version in Git, run record included, with the tag v0.1.0.

python

!git init -q -b main
!git config user.name "Your Name"
!git config user.email "you@example.com"
!git add -A && git commit -q -m "Module 1: data package, data tests, GRU vs attention race with 3 seeds"
!git tag v0.1.0
!git log --oneline --decorate && git ls-files

Code walkthrough

git init -b main starts a repository whose first branch is called main. The two config lines set the name and email Git writes on commits; use your own.
git add -A stages every file not listed in .gitignore; commit records them; tag v0.1.0 names this commit.
The last line shows the commit with its tag and the eight files Git now tracks.

You should see one commit marked (HEAD -> main, tag: v0.1.0) and eight files, including runs/m01_rnn_vs_attention.json.

To push, create an empty repository called tinyformer-lab on GitHub, then run these commands from a terminal on your own computer after downloading the folder, or in Colab after authenticating (not run here; commands from GitHub's guide "Adding locally hosted code to GitHub", checked October 2026):

bash

git remote add origin https://github.com/<your-account>/tinyformer-lab.git
git push -u origin main --tags

Code walkthrough

remote add origin records where the repository lives on GitHub. push -u origin main --tags sends the branch and the v0.1.0 tag. GitHub's guide says you must authenticate on the command line first.

Check your work

The check reruns the tests in a fresh process, reads the run record, confirms the GRU reproduces Lesson 2's 0.568, and ends with seven passes (green).

python

import json, subprocess, sys
tests = subprocess.run([sys.executable, "-m", "pytest", "-q", "tests"], capture_output=True, text=True)
record = json.load(open("runs/m01_rnn_vs_attention.json"))
runs = record["runs"]
gru11 = next(r for r in runs if r["racer"] == "gru" and r["seed"] == 11)
checks = {
    "data tests pass": tests.returncode == 0,
    "3 racers x 3 seeds recorded": sorted((r["racer"], r["seed"]) for r in runs) ==
        sorted((n, s) for n in ["gru", "gru_attention", "encoder_1layer"] for s in [11, 12, 13]),
    "parameters within 5% of the GRU": all(abs(r["parameters"] / 107296 - 1) < 0.05 for r in runs),
    "GRU seed 11 reproduces Lesson 2 (0.568)": round(gru11["accuracy"], 3) == 0.568,
    "unscored half near chance for every run": all(r["unscored_accuracy"] < 0.06 for r in runs),
    "record names its versions": {"torch", "python", "threads", "machine"} <= record.keys(),
    "tagged v0.1.0": "v0.1.0" in subprocess.run(["git", "tag"], capture_output=True, text=True).stdout,
}
for name, ok in checks.items():
    print("PASS" if ok else "FAIL", name)
print("lab check passed" if all(checks.values()) else "lab check FAILED")

Code walkthrough

subprocess.run reruns the data tests in a fresh process (sys.executable is the notebook's own Python) and keeps the exit code: 0 means every test passed.
Each entry of checks is one claim about the repository, from the full grid of runs to the tag.

You should see seven PASS lines and lab check passed.

What you added to tinyformer-lab

The repository: pyproject.toml, README.md, .gitignore and the package tinyformer/, installed in editable mode.
tinyformer/data.py with the course seed and the synthetic tasks, and tests/test_m01_data.py with four claims about them.
experiments/m01_rnn_vs_attention.py: three matched racers on three seeds each.
runs/m01_rnn_vs_attention.json: the lab's first run record, with versions, settings, curves and the unscored-half leak check.
Kofi's rule as code: a racer wins only when its seed range clears the other's.

Going further

Add "reverse" to the script and race all three on TASKS["reverse"]; predict the GRU's score first.
Shrink each racer to about 25,000 parameters and rerun: does the ranking hold at a smaller size?
Raise the seeds to five and see whether the encoder's 0.0020 lead survives a wider noise floor.