CourseNatural Language Processing: The Complete Course · Module 2: Tokenisation and Word Normalisation · part 14 of 73
Part 14 · Module 2: Tokenisation and Word Normalisation

Lesson 1: Splitting messages into words and sentences

53 min read·14 Oct 2026
Module 2 · Lesson 1 · About 40 minutes · Needs: basic Python and regular expressions (Module 1, Lesson 3)

The situation

It is Monday 14 September 2026, week 2 of the project. Every method Rosa will see from here on, from word counts to classifiers, starts from the same first step: cutting a message into pieces. Ines forwards message TW-M0806 as a test: "Spoke snapped on my Volt 2 after two rides. That's not normal wear, surely? Bought 31 May, order TW-112756. 🚲"

You need its words and its sentences. That sounds like one line of Python. This lesson shows why it is not, and how to choose a tool by measuring them all on the same messages.

The rest of this part opens with an account

Signing up is free and takes one email. Nothing to pay, and nothing renews.