Lab: Clean, validate and version the help centre

Module 2 · Lab · About 45 minutes · Needs: lessons 1 and 2 (pandas, explicit dtypes, the four gates: validate, dedupe, redact, version)

What you are building

Priya has sent the help-centre export, Dev has sent the orders table as a CSV, and the forty labelled tickets arrive as a second CSV. All three are messy in the ways the two lessons showed: an article with no category, an id turned into a number, two copies of old articles, a pasted email and phone number, a blank ticket, a resent ticket, and order numbers that a CSV reader will guess are integers. By the end of this lab the three tables have passed through the four gates and the pandas steps, and sit in one folder named by a version id: articles.parquet, orders.parquet, tickets.parquet and a manifest.json that records the row count and content hash of each. A one-page data report, printed from code, says what went in, what was held back and why. This is the clean, versioned dataset a production pipeline would hand to Modules 3, 5 and 8 (the lessons paste the same shared data so each runs alone). No model is called, so no key is needed.