Lesson 1: Measuring a classifier: beyond one accuracy number
57 min read·14 Oct 2026
Module 5 · Lesson 1 · About 50 minutes · Needs: basic Python; Module 4 (a TF-IDF and logistic regression intent model) helps, but this lesson builds the model it measures
The situation
It is Monday 5 October 2026, project week 5. Rosa has read last week's result: the intent model gets 0.613 of HARD_TEST right, the 80 hand-written messages. "Sixty-one per cent. Is that good? Which messages does it get wrong, and who suffers?"
Ines points at one: "cheers for sorting the refund so quick, really appreciate it". That is praise, but the model sent it to returns_refund. One number cannot tell Rosa that. This lesson takes it apart by intent, by kind of mistake and by language (simulated data throughout).