Lesson 1: Should you fine-tune? Decide, then measure the bar
24 min read·13 Oct 2026
Module 9 · Lesson 1 · About 50 minutes · Needs: Module 4 Lesson 2 (the frozen split and the control prompt); Module 1 Lesson 4 helps (label scoring, the Wilson interval)
The situation
It is Monday 27 July 2026, week 9. Tomasz has an idea: "Let's fine-tune a small open model for triage. We would own it, it runs on our own machine, and no ticket leaves the building." Joel asks what it costs. Maya asks the question that matters: "Better than what we have? Measured how?"
Before anyone trains, this lesson decides whether fine-tuning is the right lever, writes down how it will be judged, and measures every cheaper option. The running example is T-1025: Ruth Pell sees two charges of 144 USD and asks whether one is a duplicate. Gold label: billing.