CourseTransformers: From Attention to Trained, Served and Interpreted Models · Module 8: Pretraining · part 60 of 111
Part 60 · Module 8: Pretraining

Lesson 4: Running a pretraining job, start to finish

30 min read·13 Oct 2026
Module 8 · Lesson 4 · About 50 minutes · Needs: Module 6's lab (train with checkpoints, resume and write_manifest); Lesson 1 (the lookup probe)

The situation

Sol 421 at Aurora Station. Causal language modelling won Lesson 2's race. Kofi Mensah now wants one real pretraining job on base, run the way large teams run theirs: "Plan it before you launch, watch it, show me it survives a crash, then hand me a report."

The station's CPU servers are shared, and a life-support job can take one without warning. The running example is the job itself, runs/m08-job.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.