Part 12 · Module 1: Text as Data and Rules

Incident: A name key that changed twice

28 min read·14 Oct 2026

Module 1 · Incident file · About 30 minutes

What happened

The attack in six steps: the check passed on one form of the name (blue), a second pass changed it (amber) into someone else's (red).

In June 2013 Spotify's engineering blog, Spotify Labs, published "Creative usernames and Spotify account hijacking". It described a fault in how Spotify compared usernames. Before comparing, Spotify turned each username into a canonical form: one standard spelling, so that names differing only in case or in how a letter is encoded count as the same. The function that did this used Unicode normalisation, and it was not idempotent: applying it twice could give a different result from applying it once.

That gap was enough. A name written in unusual Unicode letters could pass the "is this name taken?" check, because its canonical form differed from every existing one. Later, when the form was applied again, it became equal to someone else's ordinary username, and an attacker could take over that account through a password reset. Spotify fixed the fault and wrote up the cause.

This is the module's subject. Lesson 2 taught normalisation and case folding as tools for matching text. Used as the key for an identity, the same tools need one more property, and a test for it.

Why it happened

An idempotent key stops after one pass (green); a non-idempotent one keeps moving (red), so two passes disagree.

A key function is meant to have a fixed point: once text has been turned into a key, turning the key into a key again must change nothing. In symbols, key(key(x)) == key(x). A function with that property is idempotent. Lesson 2's clean was tested for exactly this (clean(clean(text)) == clean(text)), because a cleaner run twice by two parts of a system must agree.

Lower case first, NFKC second: the modifier J survives the lowering, becomes a capital J, and only a second pass (red) lowers it.

Order is what breaks it. Take the modifier letter ᴶ (a small raised capital J). It has no lower-case form, so lower-casing leaves it alone. Then NFKC, the compatibility normal form from Lesson 2, replaces it with a plain capital J. The result still has a capital letter in it. One pass of "lower, then NFKC" has produced text that a second pass would change again.

The check (blue) and the lookup (amber) apply the key a different number of times, so they disagree about whose name it is (red).

The danger is in where the key is computed. The check that a name is free and the lookup that finds an account were two separate steps. If one step applies the key to the raw name and another applies it to a stored key, they compare different things. Any code that stores a key and later normalises it again, in a reset form, an import job or a second service, applies the function twice. Only an idempotent key makes that harmless.

The same failure at Tidewell

Tidewell has no customer accounts, but Ines wants a shortcut: when someone writes "where is my order?", look up their earlier orders by name. Suppose a quick key is written for it, lower case then NFKC, and stored for each new sender. This replay uses the shared data, so paste guide 06 blocks 1 to 3 first, unchanged.

python

from datetime import date, timedelta

COURSE_DATE = date(2026, 10, 14)   # facts, versions and model names checked on this day
DATA_AS_OF = date(2026, 8, 31)     # the inbox snapshot: count days to this, never to the clock
WINDOW = (date(2026, 3, 2), date(2026, 8, 30))   # 26 weeks of messages, Monday to Sunday
SEED = 20260831

EVENTS = {   # fixed dates the data and the lessons share
    "spam_wave": (date(2026, 4, 6), date(2026, 4, 19)),
    "battery_notice": date(2026, 6, 9),      # email to Volt 2 owners about pack batch VB-0426
    "eu_shipping": date(2026, 7, 1),          # Germany, Spain and France
    "summer_sale": (date(2026, 7, 15), date(2026, 7, 28)),
    "carrier_change": date(2026, 8, 10),      # ParcelNorth -> SwiftDrop
}
BATCH = "VB-0426"
STAFF = ["Rosa", "Ines", "Dev", "Callum", "Amara"]

PRODUCTS = [   # (sku, name, price in GBP, kind)
    ("TW-VOLT2-BLK", "Volt 2", 2199, "ebike"),
    ("TW-VOLT2-STP", "Volt 2 Step-Through", 2249, "ebike"),
    ("TW-HARB-GRN", "Harbour", 649, "bike"),
    ("TW-HKID-20R", "Harbour Kids 20", 329, "bike"),
    ("TW-CAUS-GRY", "Causeway", 1149, "bike"),
    ("TW-PANN-SET", "Commuter Pannier Set", 89, "accessory"),
    ("TW-BATT-500", "Volt 2 battery pack (500 Wh)", 549, "accessory"),
    ("TW-LOCK-DLK", "LockSure D-lock", 45, "accessory"),
    ("TW-LITE-BCN", "Beacon Light Set", 39, "accessory"),
    ("TW-SERV-BAS", "Basic Service", 65, "service"),
    ("TW-SERV-FUL", "Full Service", 129, "service"),
    ("TW-REPR-PUN", "Puncture Repair", 18, "service"),
    ("TW-DIAG-EBK", "E-bike Diagnostic", 45, "service"),
    ("TW-GIFT-CRD", "Gift Card", 50, "gift"),          # sold from 25 to 200 GBP; 50 is the usual
]
INTENTS = ["order_status", "returns_refund", "repair_booking", "product_question",
           "warranty_fault", "praise", "spam"]
LABELS = {   # Ines's written definitions, used for every label in the course
    "order_status": "Where an order is, delivery problems, missing items, invoices and order documents.",
    "returns_refund": "Sending something back, exchanges without a fault, cancellations, refunds and wrong charges.",
    "repair_booking": "Booking, moving or asking the price of a service or repair for wear and tear.",
    "product_question": "Questions before or after buying: sizes, stock, range, compatibility, delivery areas.",
    "warranty_fault": "A product that is faulty or arrived damaged, including the battery batch VB-0426.",
    "praise": "Thanks, compliments and good reviews, with no request that needs action.",
    "spam": "Unsolicited selling, phishing and scams.",
}

CUSTOMERS = [   # (first, last, gender, language, home town); all fictional
    ("Joan", "Hartley", "f", "en", "Clifton"), ("Oliver", "Bennett", "m", "en", "Bristol"),
    ("Amelia", "Clarke", "f", "en", "Bath"), ("Mohammed", "Khan", "m", "en", "Bristol"),
    ("Aisha", "Begum", "f", "en", "Bedminster"), ("Daniel", "O'Connor", "m", "en", "Bristol"),
    ("Priya", "Patel", "f", "en", "Swindon"), ("Arjun", "Sharma", "m", "en", "Bristol"),
    ("Grace", "Okafor", "f", "en", "Cardiff"), ("Emeka", "Eze", "m", "en", "Bristol"),
    ("Hannah", "Lewis", "f", "en", "Stroud"), ("Tom", "Hughes", "m", "en", "Gloucester"),
    ("Siobhan", "Murphy", "f", "en", "Bristol"), ("Liam", "Walsh", "m", "en", "Bath"),
    ("Mei", "Chen", "f", "en", "Bristol"), ("Wei", "Zhang", "m", "en", "Clifton"),
    ("Fatima", "Hussain", "f", "en", "Bristol"), ("Yusuf", "Ali", "m", "en", "Bedminster"),
    ("Zara", "Ahmed", "f", "en", "Taunton"), ("Jack", "Robinson", "m", "en", "Bristol"),
    ("Ella", "Morgan", "f", "en", "Cardiff"), ("Rhys", "Evans", "m", "en", "Cardiff"),
    ("Megan", "Price", "f", "en", "Weston-super-Mare"), ("Ben", "Carter", "m", "en", "Bristol"),
    ("Lucy", "Ward", "f", "en", "Bath"), ("Kofi", "Asante", "m", "en", "Bristol"),
    ("Ama", "Boateng", "f", "en", "Bristol"), ("Olusegun", "Adeyemi", "m", "en", "Swindon"),
    ("Funmi", "Adebayo", "f", "en", "Bristol"), ("Raj", "Gupta", "m", "en", "Gloucester"),
    ("Neha", "Kapoor", "f", "en", "Bristol"), ("Tomasz", "Nowak", "m", "en", "Bristol"),
    ("Agnieszka", "Kowalska", "f", "en", "Stroud"), ("Stefan", "Ionescu", "m", "en", "Bristol"),
    ("Ioana", "Popescu", "f", "en", "Bath"), ("Hiroshi", "Tanaka", "m", "en", "Bristol"),
    ("Lukas", "Becker", "m", "de", "Berlin"), ("Anna", "Schmidt", "f", "de", "Hamburg"),
    ("Jonas", "Weber", "m", "de", "München"), ("Lea", "Fischer", "f", "de", "Köln"),
    ("Mehmet", "Yılmaz", "m", "de", "Berlin"), ("Elif", "Demir", "f", "de", "Leipzig"),
    ("Felix", "Wagner", "m", "de", "Hamburg"), ("Sophie", "Müller", "f", "de", "München"),
    ("Javier", "García", "m", "es", "Madrid"), ("Lucía", "Fernández", "f", "es", "Sevilla"),
    ("Alejandro", "Martín", "m", "es", "Valencia"), ("Carmen", "Ruiz", "f", "es", "Madrid"),
    ("Mateo", "López", "m", "es", "Barcelona"), ("Valentina", "Torres", "f", "es", "Bilbao"),
    ("Youssef", "El Amrani", "m", "es", "Barcelona"), ("Nuria", "Puig", "f", "es", "Barcelona"),
    ("Hugo", "Martin", "m", "fr", "Paris"), ("Camille", "Dubois", "f", "fr", "Lyon"),
    ("Lucas", "Bernard", "m", "fr", "Nantes"), ("Chloé", "Lefèvre", "f", "fr", "Bordeaux"),
    ("Karim", "Benali", "m", "fr", "Lille"), ("Amina", "Diallo", "f", "fr", "Paris"),
    ("Théo", "Moreau", "m", "fr", "Lyon"), ("Léa", "Laurent", "f", "fr", "Nantes"),
]

print(len(PRODUCTS), "products,", len(CUSTOMERS), "customers,", len(INTENTS), "intents")
print("women:", sum(c[2] == "f" for c in CUSTOMERS), "men:", sum(c[2] == "m" for c in CUSTOMERS))
print({lang: sum(c[3] == lang for c in CUSTOMERS) for lang in ["en", "de", "es", "fr"]})
print((DATA_AS_OF - WINDOW[0]).days, "days from the first message day to DATA_AS_OF")

Code walkthrough: guide 06 block 1, unchanged: dates, products, labels and the 60 customers (CUSTOMERS).

python

import random, re

# Templates: one dict per wording. Keys en, de, es, fr hold the same message in each language
# (English-only wordings have just "en"). {UPPER} slots become gold entities; {lower} slots do not.
T = {
 "order_status": [
  {"s": "neu", "en": "Hi, I placed an order on {DATE} (order {ORDER_ID}, item: {PRODUCT}) and it still says processing. When will it ship?",
   "de": "Hallo, ich habe am {DATE} bestellt (Bestellung {ORDER_ID}, Artikel: {PRODUCT}) und der Status steht immer noch auf in Bearbeitung. Wann wird sie verschickt?",
   "es": "Hola, hice un pedido el {DATE} (pedido {ORDER_ID}, artículo: {PRODUCT}) y sigue en preparación. ¿Cuándo lo enviaréis?",
   "fr": "Bonjour, j'ai passé commande le {DATE} (commande {ORDER_ID}, article : {PRODUCT}) et elle est toujours en préparation. Quand sera-t-elle expédiée ?"},
  {"s": "neg", "en": "Order {ORDER_ID} was due on {DATE} and nothing has arrived. The {carrier} tracking has not changed for three days. Where is my parcel ({PRODUCT})?",
   "de": "Bestellung {ORDER_ID} sollte am {DATE} ankommen, aber es ist nichts da. Die Sendungsverfolgung von {carrier} hat sich seit drei Tagen nicht geändert. Wo ist mein Paket ({PRODUCT})?",
   "es": "El pedido {ORDER_ID} tenía que llegar el {DATE} y no ha llegado nada. El seguimiento de {carrier} no cambia desde hace tres días. ¿Dónde está mi paquete ({PRODUCT})?",
   "fr": "La commande {ORDER_ID} devait arriver le {DATE} et rien n'est arrivé. Le suivi {carrier} n'a pas bougé depuis trois jours. Où est mon colis ({PRODUCT}) ?"},
  {"s": "neu", "en": "Could you send me the tracking link for order {ORDER_ID}? The confirmation email never came through to {email}.",
   "de": "Könnten Sie mir den Tracking-Link für Bestellung {ORDER_ID} schicken? Die Bestätigung ist nie bei {email} angekommen.",
   "es": "¿Me podéis enviar el enlace de seguimiento del pedido {ORDER_ID}? El correo de confirmación nunca llegó a {email}.",
   "fr": "Pouvez-vous m'envoyer le lien de suivi de la commande {ORDER_ID} ? L'e-mail de confirmation n'est jamais arrivé sur {email}."},
  {"s": "neg", "from": "2026-08-10", "w": 3, "en": "Third time {carrier} has said 'attempted delivery' when I was in all day. Order {ORDER_ID}. This never happened with your old courier."},
  {"s": "neu", "en": "any update on {ORDER_ID}? need the {PRODUCT} before {DATE} if poss"},
 ],
 "returns_refund": [
  {"s": "neu", "en": "I'd like to return the {PRODUCT} from order {ORDER_ID}. It's unused and still in the box. How do I send it back?",
   "de": "Ich möchte {PRODUCT} aus Bestellung {ORDER_ID} zurückgeben. Unbenutzt und noch originalverpackt. Wie schicke ich es zurück?",
   "es": "Quiero devolver {PRODUCT} del pedido {ORDER_ID}. Está sin usar y en su caja. ¿Cómo lo envío?",
   "fr": "Je voudrais retourner {PRODUCT} de la commande {ORDER_ID}. Il n'a pas servi et il est encore dans sa boîte. Comment le renvoyer ?"},
  {"s": "neg", "en": "I sent the return for {ORDER_ID} back on {DATE} and still no refund of {MONEY}. How long does this take?",
   "de": "Ich habe die Rücksendung für {ORDER_ID} am {DATE} abgeschickt und noch keine Erstattung über {MONEY} bekommen. Wie lange dauert das?",
   "es": "Envié la devolución del pedido {ORDER_ID} el {DATE} y todavía no me habéis devuelto {MONEY}. ¿Cuánto tarda?",
   "fr": "J'ai renvoyé la commande {ORDER_ID} le {DATE} et je n'ai toujours pas reçu le remboursement de {MONEY}. Combien de temps faut-il ?"},
  {"s": "neu", "en": "The {KIDBIKE} is the wrong size for my daughter. Can I swap it or get a refund? Order {ORDER_ID}.",
   "de": "{KIDBIKE} ist für meine Tochter die falsche Größe. Kann ich umtauschen oder bekomme ich das Geld zurück? Bestellung {ORDER_ID}.",
   "es": "{KIDBIKE} es de una talla equivocada para mi hija. ¿Puedo cambiarlo o me devolvéis el dinero? Pedido {ORDER_ID}.",
   "fr": "{KIDBIKE} n'est pas à la bonne taille pour ma fille. Puis-je l'échanger ou être remboursé ? Commande {ORDER_ID}."},
  {"s": "neg", "en": "You charged me {MONEY} twice for order {ORDER_ID}. Please refund the duplicate payment."},
  {"s": "neu", "en": "want to cancel order {ORDER_ID} before it ships, changed my mind about the {PRODUCT}. refund to my card pls"},
 ],
 "repair_booking": [
  {"s": "neu", "en": "Can I book a {SERVICE} for {DATE}? It's for my bike ({BIKE}). I can drop it at the workshop in the morning.",
   "de": "Kann ich für den {DATE} einen Termin für {SERVICE} buchen? Es geht um mein Rad ({BIKE}). Ich kann es morgens in die Werkstatt bringen.",
   "es": "¿Puedo reservar {SERVICE} para el {DATE}? Es para mi bici ({BIKE}). Puedo dejarla en el taller por la mañana.",
   "fr": "Puis-je réserver {SERVICE} pour le {DATE} ? C'est pour mon vélo ({BIKE}). Je peux le déposer à l'atelier le matin."},
  {"s": "neu", "en": "My bike ({BIKE}) needs a repair: {wear}. Can I bring it in or send it to you? I live in {LOC}.",
   "de": "Mein Rad ({BIKE}) muss repariert werden: {wear}. Kann ich es vorbeibringen oder einschicken? Ich wohne in {LOC}.",
   "es": "Mi bici ({BIKE}) necesita una reparación: {wear}. ¿Puedo llevarla o enviárosla? Vivo en {LOC}.",
   "fr": "Mon vélo ({BIKE}) a besoin d'une réparation : {wear}. Puis-je l'apporter ou vous l'envoyer ? J'habite à {LOC}."},
  {"s": "neu", "en": "Please can I move my {SERVICE} booking ({ticket}) from {DATE} to the following week? Something came up.",
   "de": "Kann ich meinen Termin für {SERVICE} ({ticket}) vom {DATE} auf die Woche danach verschieben? Mir ist etwas dazwischengekommen.",
   "es": "¿Puedo cambiar mi cita de {SERVICE} ({ticket}) del {DATE} a la semana siguiente? Me ha surgido algo.",
   "fr": "Puis-je déplacer mon rendez-vous {SERVICE} ({ticket}) du {DATE} à la semaine suivante ? J'ai un empêchement."},
  {"s": "neu", "en": "hiya, how much is a {SERVICE} and can u fit me in {DATE}? bike is a {BIKE}, cheers"},
  {"s": "neu", "en": "I'd like to book the {SERVICE} for my {BIKE}. Is {MONEY} still the price? Any day after {DATE} works."},
 ],
 "product_question": [
  {"s": "neu", "en": "Does the {BIKE} come in a smaller frame? I'm 158 cm tall.",
   "de": "Gibt es {BIKE} auch mit einem kleineren Rahmen? Ich bin 1,58 m groß.",
   "es": "¿{BIKE} viene en una talla de cuadro más pequeña? Mido 1,58 m.",
   "fr": "{BIKE} existe-t-il avec un cadre plus petit ? Je mesure 1,58 m."},
  {"s": "neu", "en": "What range does the {EBIKE} get on one charge, and is the {ACCESSORY} compatible with it?",
   "de": "Welche Reichweite hat {EBIKE} mit einer Akkuladung, und passt {ACCESSORY} dazu?",
   "es": "¿Qué autonomía tiene {EBIKE} con una carga, y es compatible con {ACCESSORY}?",
   "fr": "Quelle autonomie a {EBIKE} avec une charge, et {ACCESSORY} est-il compatible ?"},
  {"s": "neu", "en": "Is the {PRODUCT} in stock? The website says {MONEY} but I can't add it to the basket.",
   "de": "Ist {PRODUCT} auf Lager? Auf der Website steht {MONEY}, aber ich kann es nicht in den Warenkorb legen.",
   "es": "¿Tenéis {PRODUCT} en stock? En la web pone {MONEY} pero no puedo añadirlo a la cesta.",
   "fr": "Est-ce que {PRODUCT} est en stock ? Le site indique {MONEY} mais je n'arrive pas à l'ajouter au panier."},
  {"s": "neu", "en": "Do you deliver to {LOC}? And do bikes arrive fully built or do I need to assemble them?"},
  {"s": "neu", "en": "quick q: whats the difference between the {SERVICE} and the {SERVICE2}?"},
 ],
 "warranty_fault": [
  {"s": "neg", "en": "My e-bike ({EBIKE}) is only four months old and {fault}. Is this covered by the warranty?",
   "de": "Mein E-Bike ({EBIKE}) ist erst vier Monate alt und {fault}. Ist das von der Garantie abgedeckt?",
   "es": "Mi bici eléctrica ({EBIKE}) solo tiene cuatro meses y {fault}. ¿Lo cubre la garantía?",
   "fr": "Mon vélo électrique ({EBIKE}) n'a que quatre mois et {fault}. Est-ce couvert par la garantie ?"},
  {"s": "neg", "from": "2026-06-09", "w": 4, "en": "I got your email about battery batch {batch}. My pack has that code and {fault}. What happens now?",
   "de": "Ich habe Ihre E-Mail zur Akku-Charge {batch} bekommen. Mein Akku hat diesen Code und {fault}. Wie geht es jetzt weiter?",
   "es": "Recibí vuestro correo sobre el lote de baterías {batch}. Mi batería tiene ese código y {fault}. ¿Qué hago ahora?",
   "fr": "J'ai reçu votre e-mail sur le lot de batteries {batch}. Ma batterie porte ce code et {fault}. Que se passe-t-il maintenant ?"},
  {"s": "neg", "en": "The {PRODUCT} arrived with {damage}. I've attached photos. Can you send a replacement?",
   "de": "{PRODUCT} kam mit {damage} an. Fotos sind im Anhang. Können Sie Ersatz schicken?",
   "es": "{PRODUCT} llegó con {damage}. Adjunto fotos. ¿Podéis enviarme uno nuevo?",
   "fr": "{PRODUCT} est arrivé avec {damage}. Je joins des photos. Pouvez-vous m'en envoyer un autre ?"},
  {"s": "neg", "en": "Spoke snapped on my {BIKE} after two rides. That's not normal wear, surely? Bought {DATE}, order {ORDER_ID}."},
  {"s": "neg", "en": "{EBIKE} battery dead again. Second pack in a month. I want this sorted properly this time."},
 ],
 "praise": [
  {"s": "pos", "en": "Just wanted to say thank you to {PERSON} in the workshop. My bike ({BIKE}) rides like new after the {SERVICE}.",
   "de": "Ich wollte mich nur bei {PERSON} in der Werkstatt bedanken. Mein Rad ({BIKE}) fährt sich nach {SERVICE} wie neu.",
   "es": "Solo quería dar las gracias a {PERSON} del taller. Mi bici ({BIKE}) va como nueva después de {SERVICE}.",
   "fr": "Je voulais juste remercier {PERSON} de l'atelier. Mon vélo ({BIKE}) roule comme neuf après {SERVICE}."},
  {"s": "pos", "en": "Love my new bike ({BIKE}). Fast delivery to {LOC} and it was easy to set up. Five stars.",
   "de": "Ich liebe mein neues Rad ({BIKE}). Schnelle Lieferung nach {LOC} und leicht aufzubauen. Fünf Sterne.",
   "es": "Me encanta mi nueva bici ({BIKE}). Entrega rápida a {LOC} y fácil de montar. Cinco estrellas.",
   "fr": "J'adore mon nouveau vélo ({BIKE}). Livraison rapide à {LOC} et facile à monter. Cinq étoiles."},
  {"s": "pos", "en": "Great service from {PERSON}, who sorted my problem in one email. Thank you!",
   "de": "Super Service von {PERSON}, das Problem war mit einer E-Mail gelöst. Vielen Dank!",
   "es": "Un servicio genial de {PERSON}, que me lo resolvió en un solo correo. ¡Gracias!",
   "fr": "Super service de la part de {PERSON}, qui a réglé mon problème en un seul e-mail. Merci !"},
  {"s": "pos", "en": "the {ACCESSORY} is brilliant, fits perfectly. cheers"},
  {"s": "pos", "en": "Honestly the best bike shop in {LOC}. Friendly, quick and fair prices."},
 ],
 "spam": [
  {"s": "neu", "en": "Dear partner, we offer guaranteed first-page search ranking for your online store. Reply today for a {MONEY} discount."},
  {"s": "neu", "en": "Exclusive partnership opportunity: invest in our crypto bike-sharing token and earn 300% returns. Contact {email} now."},
  {"s": "neu", "en": "Your mailbox storage is full. Verify your account within 24 hours at http://mail-verify.example/login to avoid suspension."},
  {"s": "neu", "en": "Hello, I am an influencer with 50k followers. Send me a free {BIKE} and I will post about your brand."},
  {"s": "neu", "en": "Congratulations! You have been selected for a {MONEY} gift voucher. Click http://prize-claim.example to claim."},
 ],
}
PHRASES = {   # paired fillers for the {lower} slots: (en, de, es, fr)
 "fault": [("the battery will not charge", "der Akku lädt nicht mehr", "la batería no carga", "la batterie ne se charge plus"),
           ("the motor cuts out on hills", "der Motor setzt am Berg aus", "el motor se para en las cuestas", "le moteur se coupe dans les montées"),
           ("the display shows error E07", "das Display zeigt Fehler E07", "la pantalla muestra el error E07", "l'écran affiche l'erreur E07"),
           ("the battery runs flat very quickly", "der Akku ist sehr schnell leer", "la batería se agota muy rápido", "la batterie se vide très vite")],
 "wear": [("the back tyre has a puncture", "der Hinterreifen hat einen Platten", "la rueda trasera está pinchada", "le pneu arrière est crevé"),
          ("the gears keep slipping", "die Gangschaltung springt ständig", "los cambios saltan todo el rato", "les vitesses sautent sans arrêt"),
          ("the brake pads are worn out", "die Bremsbeläge sind abgenutzt", "las pastillas de freno están gastadas", "les plaquettes de frein sont usées"),
          ("the chain squeaks", "die Kette quietscht", "la cadena chirría", "la chaîne grince")],
 "damage": [("a cracked mudguard", "einem gebrochenen Schutzblech", "un guardabarros roto", "un garde-boue fendu"),
            ("a bent front wheel", "einem verbogenen Vorderrad", "la rueda delantera doblada", "une roue avant voilée"),
            ("a broken clip", "einem kaputten Clip", "un clip roto", "un clip cassé")],
}
SUBJECTS = {"order_status": ["Where is my order?", "Wo bleibt meine Bestellung?", "¿Dónde está mi pedido?", "Où est ma commande ?"],
            "returns_refund": ["Return and refund", "Rücksendung", "Devolución", "Retour"],
            "repair_booking": ["Booking a service", "Werkstatttermin", "Cita en el taller", "Rendez-vous atelier"],
            "product_question": ["Product question", "Frage zum Produkt", "Pregunta sobre un producto", "Question sur un produit"],
            "warranty_fault": ["Fault with my bike", "Defekt", "Avería", "Panne"],
            "praise": ["Thank you!", "Danke!", "¡Gracias!", "Merci !"],
            "spam": ["Partnership opportunity"] * 4}
FORM_TOPIC = {"order_status": "Order enquiry", "returns_refund": "Returns", "repair_booking": "Workshop",
              "product_question": "Product question", "warranty_fault": "Faulty item", "praise": "Feedback", "spam": "Other"}
CHANNELS = {"order_status": (5, 4, 1), "returns_refund": (6, 4, 0), "repair_booking": (4, 6, 0),   # email, web form, review
            "product_question": (4, 6, 0), "warranty_fault": (5, 3, 2), "praise": (3, 1, 6), "spam": (1, 0, 0)}
LANGS = ["en", "de", "es", "fr"]
MONTHS = {"en": "January February March April May June July August September October November December",
          "de": "Januar Februar März April Mai Juni Juli August September Oktober November Dezember",
          "es": "enero febrero marzo abril mayo junio julio agosto septiembre octubre noviembre diciembre",
          "fr": "janvier février mars avril mai juin juillet août septembre octobre novembre décembre"}
TOWNS = {"en": ["Bristol", "Bath", "Clifton", "Cardiff", "Stroud"], "de": ["Berlin", "Hamburg", "München", "Köln"],
         "es": ["Madrid", "Barcelona", "Valencia", "Sevilla"], "fr": ["Paris", "Lyon", "Nantes", "Lille"]}
SPAMMERS = [("Growth Partners", "offers@growth-partners.example"), ("Crypto Ride", "team@cryptoride.example"),
            ("Mail Admin", "admin@mail-verify.example"), ("Prize Desk", "win@prize-claim.example")]


def fmt_date(d, lang, rng):
    m = MONTHS[lang].split()[d.month - 1]
    styles = {"en": [f"{d.day} {m}", f"{d.day:02d}/{d.month:02d}", f"{m} {d.day}{'th' if 10 < d.day < 14 else {1: 'st', 2: 'nd', 3: 'rd'}.get(d.day % 10, 'th')}"],
              "de": [f"{d.day}. {m}", f"{d.day:02d}.{d.month:02d}."], "es": [f"{d.day} de {m}", f"{d.day}/{d.month}"],
              "fr": [f"{d.day} {m}", f"{d.day:02d}/{d.month:02d}/{d.year}"]}[lang]
    return rng.choice(styles) if rng else styles[0]


def fmt_money(gbp, lang, rng):
    styles = {"en": [f"£{gbp:,}", f"£{gbp:,}.00", f"{gbp} pounds"], "de": [f"{gbp:,} £".replace(",", "."), f"{gbp} Pfund"],
              "es": [f"{gbp:,} libras".replace(",", "."), f"{gbp} £"], "fr": [f"{gbp:,} £".replace(",", " "), f"{gbp} livres"]}[lang]
    return rng.choice(styles) if rng else styles[0]


def typo(word, rng):
    i = rng.randrange(1, len(word) - 1)
    return rng.choice([word[:i] + word[i + 1] + word[i] + word[i + 2:], word[:i] + word[i + 1:], word[:i] + word[i] + word[i:]])


print(sum(len(v) for v in T.values()), "wordings,", sum("de" in t for v in T.values() for t in v), "of them in all four languages")

Code walkthrough: guide 06 block 2, unchanged: the wordings.

python

import html, unicodedata

POOLS = {"PRODUCT": ["bike", "ebike", "accessory"], "BIKE": ["bike", "ebike"], "EBIKE": ["ebike"],
         "ACCESSORY": ["accessory"], "SERVICE": ["service"], "SERVICE2": ["service"], "KIDBIKE": ["kids"]}
EXTRA = {"greet": {"en": ["Hi,", "Hello,", "Hello Tidewell,", "Dear Sir or Madam,"], "de": ["Hallo,", "Guten Tag,"],
                   "es": ["Hola,", "Buenos días,"], "fr": ["Bonjour,", "Madame, Monsieur,"]},
         "sign": {"en": ["Thanks,", "Many thanks,", "Cheers,"], "de": ["Danke,", "Viele Grüße,"],
                  "es": ["Gracias,", "Un saludo,"], "fr": ["Merci,", "Cordialement,"]},
         "phone": {"en": "You can call me on ", "de": "Sie erreichen mich unter ", "es": "Me podéis llamar al ",
                   "fr": "Vous pouvez m'appeler au "},
         "quote": {"en": ("On ", ", Tidewell Cycles <help@tidewell.example> wrote:"), "de": ("Am ", " schrieb Tidewell Cycles <help@tidewell.example>:"),
                   "es": ("El ", ", Tidewell Cycles <help@tidewell.example> escribió:"), "fr": ("Le ", ", Tidewell Cycles <help@tidewell.example> a écrit :")},
         "again": {"en": "Sorry, sending this again as I have not heard back. ", "de": "Entschuldigung, ich schicke das noch einmal. ",
                   "es": "Perdón, lo envío otra vez porque no he recibido respuesta. ", "fr": "Désolé, je renvoie ce message car je n'ai pas eu de réponse. "}}
LABEL_OF = {"PRODUCT": "PRODUCT", "BIKE": "PRODUCT", "EBIKE": "PRODUCT", "ACCESSORY": "PRODUCT", "SERVICE": "PRODUCT",
            "SERVICE2": "PRODUCT", "KIDBIKE": "PRODUCT", "ORDER_ID": "ORDER_ID", "DATE": "DATE", "MONEY": "MONEY",
            "LOC": "LOC", "PERSON": "PERSON"}


def fill(template, values, lang, rng):
    """Template text -> list of (text, label or None) pieces, formatting dates and money for `lang`."""
    pieces = []
    for part in re.split(r"(\{\w+\})", template):
        if not (part.startswith("{") and part.endswith("}")):
            pieces.append((part, None)) if part else None
            continue
        slot = part[1:-1]
        v = values[slot]
        if slot == "DATE":
            v = fmt_date(v, lang, rng)
        elif slot == "MONEY":
            v = fmt_money(v, lang, rng)
        elif slot in ("fault", "wear", "damage"):
            v = PHRASES[slot][v][LANGS.index(lang)]
        pieces.append((v, LABEL_OF.get(slot)))
    return pieces


def email_of(first, last):
    """'Mehmet', 'Yılmaz' -> 'mehmet.yilmaz@...'; one fixed address per customer, on a reserved .example domain."""
    plain = unicodedata.normalize("NFKD", f"{first}.{last}".lower().replace("ı", "i").replace(" ", "").replace("'", ""))
    plain = "".join(ch for ch in plain if ch.isascii())
    return plain + "@" + ["post.example", "mailbox.example", "inbox.example"][len(first + last) % 3]


def make_messages(n=1800, seed=SEED):
    rng = random.Random(seed)
    days = [WINDOW[0] + timedelta(i) for i in range((WINDOW[1] - WINDOW[0]).days + 1)]
    base = {"order_status": 26, "returns_refund": 16, "repair_booking": 17, "product_question": 18,
            "warranty_fault": 11, "praise": 6, "spam": 6}

    def weights(d):
        w = dict(base)
        w["spam"] *= 8 if EVENTS["spam_wave"][0] <= d <= EVENTS["spam_wave"][1] else 1
        w["warranty_fault"] *= 3.5 if EVENTS["battery_notice"] <= d <= date(2026, 7, 31) else 2 if d.month == 8 else 1
        w["order_status"] *= (2.5 if EVENTS["summer_sale"][0] <= d <= EVENTS["summer_sale"][1] else 1) * (1.6 if d >= EVENTS["carrier_change"] else 1)
        w["returns_refund"] *= 1.3 if date(2026, 7, 29) <= d <= date(2026, 8, 15) else 1
        vol = (0.6 if d.weekday() >= 5 else 1.0) * (1.5 if d >= EVENTS["eu_shipping"] else 1.0)
        return [w[i] * vol for i in INTENTS]

    W = [weights(d) for d in days]
    n_dup = round(n * 0.03)
    picks = sorted(rng.choices(range(len(days)), weights=[sum(w) for w in W], k=n - n_dup))
    by_kind = {k: [p for p in PRODUCTS if p[3] == k] for k in ["bike", "ebike", "accessory", "service"]}
    by_kind["kids"] = [p for p in PRODUCTS if p[1] == "Harbour Kids 20"]
    out = []
    for di in picks:
        d = days[di]
        intent = rng.choices(INTENTS, weights=W[di])[0]
        lang = "en" if intent == "spam" else rng.choices(LANGS, weights=[95, 2, 1.5, 1.5] if d < EVENTS["eu_shipping"] else [58, 16, 14, 12])[0]
        options = [t for t in T[intent] if lang in t and date.fromisoformat(t.get("from", "2026-01-01")) <= d]
        tmpl = rng.choices(options, weights=[t.get("w", 1) for t in options])[0]
        if intent == "spam":
            cust, email = rng.choice(SPAMMERS)
            first, last, gender, town = cust, "", None, None
        else:
            first, last, gender, _, town = rng.choice([c for c in CUSTOMERS if c[3] == lang])
            email = email_of(first, last)
        v = {}
        for slot, kinds in POOLS.items():
            pool = [p for k in kinds for p in by_kind[k]]
            pick = rng.choice([p for p in pool if p[1] != v.get("SERVICE")])
            v[slot] = pick[1]
            v[slot + "_sku"], v[slot + "_price"] = pick[0], pick[2]
        main = next((s for s in POOLS if "{" + s + "}" in tmpl["en"]), None)
        future = intent == "repair_booking" or "before {DATE}" in tmpl["en"]
        v["DATE"] = d + timedelta(rng.randint(2, 14)) if future else d - timedelta(rng.randint(3, 20))
        v["MONEY"] = v[main + "_price"] if main else rng.choice([50, 100, 250, 500])
        v["ORDER_ID"] = f"TW-{100000 + di * 120 + rng.randrange(120)}"
        lives_there = "live in {LOC}" in tmpl["en"] or "shop in {LOC}" in tmpl["en"]
        v["LOC"] = town if lives_there and town else rng.choice(TOWNS[lang])
        v["PERSON"] = rng.choice(["Dev", "Dev", "Ines", "Callum"]) if intent == "praise" else first
        v.update(carrier="SwiftDrop" if d >= EVENTS["carrier_change"] else "ParcelNorth", email=email, batch=BATCH,
                 ticket=f"RJ-2026-{300 + di * 2 + rng.randrange(3):04d}", fault=rng.randrange(4), wear=rng.randrange(4),
                 damage=rng.randrange(3))
        core = fill(tmpl[lang], v, lang, rng)
        en_ref = "".join(t for t, _ in fill(tmpl["en"], v, "en", None)) if lang != "en" else None
        channel = rng.choices(["email", "web form", "review"], weights=CHANNELS[intent])[0]
        channel = "web form" if channel == "review" and tmpl["s"] == "neu" else channel   # reviews praise or complain
        pieces, g = list(core), 0
        if channel == "email":
            if rng.random() < 0.7:
                pieces, g = [(rng.choice(EXTRA["greet"][lang]) + "\n\n", None)] + pieces, 1
            if rng.random() < 0.15:
                pieces += [("\n" + EXTRA["phone"][lang] + rng.choice(["07700 900", "07700900", "+44 7700 900"]) + f"{rng.randrange(1000):03d}.", None)]
            if lang == "en" and intent in ("repair_booking", "order_status") and rng.random() < 0.3:
                pieces += [(f"\nMy postcode is BS{rng.randint(1, 16)} {rng.randint(1, 9)}{rng.choice('ABDEFGHJLNPQRSTUWXYZ')}{rng.choice('ABDEFGHJLNPQRSTUWXYZ')}.", None)]
            if rng.random() < 0.7 and intent != "spam":
                pieces += [("\n\n" + rng.choice(EXTRA["sign"][lang]) + "\n", None), (first if rng.random() < 0.6 else f"{first} {last}", "PERSON")]
                if rng.random() < 0.15:
                    pieces += [("\n" + {"en": "Sent from my phone", "de": "Von meinem Handy gesendet", "es": "Enviado desde mi móvil", "fr": "Envoyé de mon téléphone"}[lang], None)]
            if rng.random() < 0.25 and intent != "spam":
                q = EXTRA["quote"][lang]
                prev = fmt_date(d - timedelta(rng.randint(1, 6)), lang, None) + (" de 2026" if lang == "es" else " 2026")
                pieces += [("\n\n> " + q[0], None), (prev, "DATE"), (q[1] + "\n> Thanks for your message. We will reply within one working day.", None)]
        r = rng.random
        if r() < 0.15:                                          # typos
            for _ in range(rng.randint(1, 2)):
                i = rng.randrange(len(pieces))
                if pieces[i][1] is None:
                    words = pieces[i][0].split(" ")
                    j = rng.randrange(len(words))
                    if len(words[j]) >= 5 and words[j].isalpha():
                        words[j] = typo(words[j], rng)
                        pieces[i] = (" ".join(words), None)
        case = r()
        if case < 0.03:
            pieces = [(t.upper(), l) for t, l in pieces]
        elif case < 0.09:
            pieces = [(t.lower(), l) for t, l in pieces]
        if r() < 0.08:                                          # emoji after the main text
            pieces.insert(g + len(core), (" " + rng.choice(["😡", "😤", "🚲"] if tmpl["s"] == "neg" else ["🙂", "👍", "🚲"]), None))
        if r() < 0.05:                                          # odd whitespace
            i = next((i for i, (t, l) in enumerate(pieces) if l is None and " " in t), None)
            if i is not None:
                t = pieces[i][0]
                pieces[i] = (t.replace(" ", " ", 1) if r() < 0.5 else t.replace(" ", " ", 1), None)
        if channel == "web form" and r() < 0.6:                 # HTML from the form
            pieces = [("<p>", None)] + [(html.escape(t).replace("\n", "<br>\n"), None) if l is None else (t, l) for t, l in pieces] + [("</p>", None)]
        if r() < 0.04:                                          # mojibake: UTF-8 bytes read as cp1252
            try:
                pieces = [(t.encode("utf-8").decode("cp1252"), l) for t, l in pieces]
            except UnicodeDecodeError:
                pass
        body, spans, pos = "", [], 0
        for t, l in pieces:
            if l:
                spans.append((pos, pos + len(t), l))
            body, pos = body + t, pos + len(t)
        sent = tmpl["s"]
        stars = None
        if channel == "review":
            stars = {"pos": rng.choice([4, 5]), "neu": 3, "neg": rng.choice([1, 2])}[sent]
            if r() < 0.08:
                stars = {"pos": rng.choice([1, 2]), "neu": rng.choice([1, 5]), "neg": 5}[sent]
        subject = {"email": SUBJECTS[intent][LANGS.index(lang)], "web form": "Web form: " + FORM_TOPIC[intent],
                   "review": "Review: " + (v[main] if main else "Tidewell Cycles")}[channel]
        out.append({"date": d.isoformat(), "channel": channel, "language": lang, "customer": f"{first} {last}".strip(),
                    "gender": gender, "email": email, "subject": subject, "body": body, "intent": intent, "sentiment": sent,
                    "stars": stars, "entities": spans, "sku": v[main + "_sku"] if main else None,
                    "order_id": v["ORDER_ID"] if "{ORDER_ID}" in tmpl["en"] else None, "duplicate_of": None,
                    "core": "".join(t for t, _ in core), "en_ref": en_ref, "label_flipped": False})
    originals = [m for m in out if m["intent"] != "spam"]
    for m in rng.sample(originals, n_dup):                      # near-duplicates: the same customer writes again
        again, cut = EXTRA["again"][m["language"]], 3 if m["body"].startswith("<p>") else 0   # inside the form's HTML
        dup = dict(m, body=m["body"][:cut] + again + m["body"][cut:], entities=[(a + len(again), b + len(again), l) for a, b, l in m["entities"]],
                   date=min(date.fromisoformat(m["date"]) + timedelta(rng.randint(0, 2)), WINDOW[1]).isoformat(), duplicate_of=m)
        out.append(dup)
    out.sort(key=lambda m: (m["date"], m["duplicate_of"] is not None))
    for i, m in enumerate(out, 1):
        m["id"] = f"TW-M{i:04d}"
    for m in out:
        if m["duplicate_of"] is not None:
            m["duplicate_of"] = m["duplicate_of"]["id"]
    for m in rng.sample(out, round(n * 0.02)):                  # deliberate wrong labels, for error analysis
        m["intent"], m["label_flipped"] = rng.choice([i for i in INTENTS if i != m["intent"]]), True
    split_of = {}
    for intent in INTENTS:                                      # stratified 70 / 15 / 15, duplicates follow their original
        ids = [m["id"] for m in out if m["intent"] == intent and m["duplicate_of"] is None]
        rng.shuffle(ids)
        a, b = round(len(ids) * 0.70), round(len(ids) * 0.85)
        split_of.update({x: "train" if k < a else "dev" if k < b else "test" for k, x in enumerate(ids)})
    for m in out:
        m["split"] = split_of[m["duplicate_of"] or m["id"]]
        m["period"] = "before_july" if m["date"] < "2026-07-01" else "from_july"
    return out


MESSAGES = make_messages()
print(len(MESSAGES), "messages;", MESSAGES[0]["date"], "to", MESSAGES[-1]["date"])
print(MESSAGES[0]["id"], "|", MESSAGES[0]["subject"], "|", MESSAGES[0]["body"][:80].replace("\n", " "))

Code walkthrough: guide 06 block 3, unchanged: the generator, which makes MESSAGES.

Now build the name keys and Joan Hartley's orders.

python

import unicodedata


def naive_key(name):
    """The quick way: lower case, then NFKC. It looks harmless."""
    return unicodedata.normalize("NFKC", name.lower())


KEYS = {naive_key(f"{first} {last}"): f"{first} {last}" for first, last, *_ in CUSTOMERS}
orders = {}
for m in MESSAGES:
    if m["order_id"] and m["gender"]:                     # customers only: spam senders have no gender
        orders.setdefault(naive_key(m["customer"]), set()).add(m["order_id"])
print(len(KEYS), "customer keys")
print("joan hartley:", len(orders["joan hartley"]), "orders, e.g.", sorted(orders["joan hartley"])[:3])

Code walkthrough

naive_key lower-cases a name, then applies NFKC. KEYS maps each of the 60 customers' keys to their name.
orders collects every order number each customer has written about, under their key. setdefault makes an empty set the first time a key is seen.

Output

text

60 customer keys
joan hartley: 12 orders, e.g. ['TW-100531', 'TW-105408', 'TW-106008']

At Tidewell the newcomer is accepted (blue) and the second pass hands them Mrs Hartley's 12 orders (red).

A new sender signs as ᴶoan Hartley, with the modifier letter in front. The system checks the name is free, stores its key, and later the order lookup normalises that stored key once more.

python

newcomer = "ᴶoan Hartley"                  # U+1D36 MODIFIER LETTER CAPITAL J, then "oan Hartley"
stored = naive_key(newcomer)
print(repr(stored), "| already taken:", stored in KEYS)
KEYS[stored] = newcomer                          # registered as a new customer
looked_up = naive_key(stored)                    # the lookup normalises the stored key again
print(repr(looked_up), "->", len(orders.get(looked_up, ())), "orders, e.g.", sorted(orders[looked_up])[:3])

Code walkthrough

"ᴶ" writes the modifier letter as an escape, so you can see it in the code.
stored is the key after one pass: 'Joan hartley', with a capital J. It is not in KEYS, so the newcomer is accepted.
looked_up is the key after a second pass: the capital is now lowered, and it is Joan's key.

Output

text

'Joan hartley' | already taken: False
'joan hartley' -> 12 orders, e.g. ['TW-100531', 'TW-105408', 'TW-106008']

A stranger's "where is my order?" would now be answered with Mrs Hartley's twelve orders. Nothing was hacked: two harmless-looking steps disagreed.

The fix

The fix: Lesson 2's fold reaches the final key in one pass, so the newcomer is refused at the door (green).

The fix has two parts. Use a key that is idempotent: Lesson 2's fold, which applies NFKC first and case-folds after. And check the property where it matters, when a key is created, so that the "is it taken?" check and the lookup compare the same thing.

python

def fold(text):
    """Lesson 2's inbox/clean.py: a matching key, NFKC first, then casefold."""
    return unicodedata.normalize("NFKC", text).casefold()


def register(name, keys):
    key = fold(name)
    if fold(key) != key:
        raise ValueError(f"unstable key for {name!r}")
    if key in keys:
        return f"refused: {name!r} matches {keys[key]!r}"
    keys[key] = name
    return f"registered {key!r}"


SAFE = {fold(f"{first} {last}"): f"{first} {last}" for first, last, *_ in CUSTOMERS}
print(register(newcomer, SAFE))
print(register("Joan Harley", SAFE))

Code walkthrough

fold is Lesson 2's, unchanged. NFKC turns ᴶ into J first, then casefold lowers it, so one pass reaches the final key.
register makes the key, refuses it if a second pass would change it, and refuses a key that is already taken. The same key is used for the check and for storage, so there is nothing left to normalise later.
"Joan Harley" (no t) is a different person, and is registered.

Output

text

refused: 'ᴶoan Hartley' matches 'Joan Hartley'
registered 'joan harley'

How you would have caught it

A test over every code point finds 629 characters the naive key changes twice (red) and none for fold (green).

One test would have found it before any customer did: apply the key twice to every Unicode character and compare. There are about 1.1 million code points, and the loop takes well under a second.

python

def unstable(key):
    """Code points c where key(key(c)) differs from key(c)."""
    chars = (chr(cp) for cp in range(0x110000) if not 0xD800 <= cp < 0xE000)   # skip surrogates
    return [c for c in chars if key(key(c)) != key(c)]


bad = unstable(naive_key)
print("naive_key:", len(bad), "unstable code points, e.g.", [f"U+{ord(c):04X} {unicodedata.name(c)}" for c in bad[3:5]])
print("fold     :", len(unstable(fold)), "unstable code points")

Code walkthrough

range(0x110000) is every Unicode code point. The surrogates, D800 to DFFF, are reserved halves of pairs and cannot stand alone, so they are skipped.
unstable keeps every single character a key changes on its second pass. Put the same comparison in tests/ as a test asserting the list is empty, and run it whenever the key function changes.

Output

text

naive_key: 629 unstable code points, e.g. ['U+1D2C MODIFIER LETTER CAPITAL A', 'U+1D2D MODIFIER LETTER CAPITAL AE']
fold     : 0 unstable code points

The test checks single characters only; sequences of characters can combine in ways it does not cover, so add your own customer names and the attack strings you have seen. Python's unicodedata follows a fixed Unicode version that changes with Python releases, which is one more reason to keep the test in the suite rather than run it once.

Key points

A key used for identity must be idempotent: key(key(x)) == key(x). Test it over every code point, not on a few names.
Order matters: NFKC first, then case folding. "Lower, then NFKC" fails on 629 characters.
Compute a key once, store it, and use the same stored key for the "is it taken?" check and the lookup. Normalising a stored key again is how two steps start to disagree.
Lesson 2's fold is a matching key; this case is why it was written in that order and why clean is tested for idempotence.

Sources

Spotify Labs, "Creative usernames and Spotify account hijacking", Spotify engineering blog, June 2013.
Python Software Foundation, "unicodedata: Unicode Database", Python 3 documentation, https://docs.python.org/3/library/unicodedata.html.
Unicode Consortium, "Unicode Standard Annex #15: Unicode Normalization Forms", https://www.unicode.org/reports/tr15/.