Lesson 3: Stop words, stemming and lemmatisation, measured
42 min read·14 Oct 2026
Module 2 · Lesson 3 · About 40 minutes · Needs: basic Python; Lesson 1 of this module helps (what a token is) but is not required
The situation
It is Monday 14 September 2026, project week 2. Ines types "charging" into the inbox search, looking for Volt 2 batteries that will not charge. It finds nothing: the messages say "charge" and "charged", and the search matches only the exact word.
Rosa asks whether the computer can treat "charging", "charged" and "charge" as one word. It can, in two ways, and there is an older habit of throwing away little words such as "the" and "not". This lesson measures all three on the Tidewell inbox (simulated data): where each helps, and where it quietly does damage.