Lesson 2: TF-IDF: weighting words and comparing messages
41 min read·14 Oct 2026
Module 4 · Lesson 2 · About 45 minutes · Needs: basic Python; Lesson 1 of this module (turning messages into a table of word counts) helps but is not required
The situation
It is Monday 28 September 2026, project week 4. On Thursday 25 June Mei Chen wrote: "Volt 2 Step-Through battery dead again. Second pack in a month. I want this sorted properly this time." The next day she sent it again, starting "Sorry, sending this again as I have not heard back." Ines answered both, separately.
Rosa asks two things. Can the inbox spot a message that repeats an earlier one? And which words mark each kind of message? Plain word counts cannot answer either: the most frequent word in Mei's message is "this". This lesson weights words by how rare they are, compares messages by direction, and tries both on the Tidewell inbox (simulated data).