Overview
The problem. Every support bot, KYC assistant and clinic chatbot in India sends user text to a model. That text carries Aadhaar numbers, PAN, UPI ids, phone numbers and names, typed by people who were never told where it goes. Most teams either send it as is, or bolt on a regex that flags every 12-digit number and is switched off within a week. Neither is defensible under the Digital Personal Data Protection Act, 2023, and neither survives the first audit question: how do you know what it catches?
What you build. A firewall that sits between an application and any LLM: it finds personal identifiers in a message, masks or tokenises them before the prompt leaves the machine, puts the real values back into the answer, and writes logs that never contain the identifier. You measure it the whole way: precision and recall per entity on a labelled set you generate, an adversarial set that tries to slip past it, and a cost and latency table for regex-only against a model-assisted pass.
What you have at the end. A repository with a scorer, five recognisers with a checksum where one exists, a reversible token vault, a hybrid name detector you have compared honestly against the regex-only path, an evasion set with before and after recall, a p50/p95 table, and a README that maps every control to the clause of the Act it answers. One number, one table, one spec. That is what a hiring manager or an auditor opens first.
Before you start, can you answer these? Every milestone assumes you can. Each one is covered by something on this site.
- What is the difference between precision and recall, and which one does a firewall have to favour? (AI Security kit)
- Why does a RAG or chat app leak personal data twice, once in the prompt and once in the trace? (LLMOps kit, RAG course)
- What does a Data Fiduciary owe under the DPDP Act when it processes a customer's Aadhaar? (Finance RAG and Agents kit, Healthcare RAG and Agents kit)
- Can you write a Python regex with a named group, a word boundary and a negative lookbehind without looking it up? (Python course)
If two or more are a no, read the linked kit first. The free opening part has the full readiness check with answers.
Pairs with: AI Security, Finance RAG and Agents, Healthcare RAG and Agents, LLMOps.
What you will learn
- Build a labelled evaluation set for personal data detection and score a detector on precision, recall and F1 per entity, with span-level matching.
- Write recognisers for Aadhaar (Verhoeff check digit), PAN (holder-type letter), Indian mobile numbers, IFSC and UPI ids, and explain where each one is certain and where it guesses.
- Use context words and confidence thresholds to separate an Aadhaar from a phone from a bank account, and read a precision-recall trade-off without wishful thinking.
- Compare a regex-only detector with a model-assisted one (Presidio and spaCy, optional local LLM) on names and addresses, with the cost and latency of each.
- Put a reversible token vault between an app and an LLM so the model never sees the identifier and the user never sees the token, and keep logs that would pass a breach review.
- Attack your own firewall with spaced, spelled-out and OCR-style variants, measure the recall drop, and fix it.
- Write the README as a product spec that maps each control to the DPDP Act, 2023 and the DPDP Rules, 2025, with a limitations section an auditor would believe.
Included with the project
- 9 written parts, yours for good
- 1h 8m of reading, measured not estimated
- Written for the intermediate level
- Every future revision included
- UPI, cards and netbanking
Prerequisites
- Python 3.11 or newer, and comfort with
re, dataclasses,argparseand pytest. Milestones 1 to 3 and 5 to 7 are standard library plus pytest. - Precision, recall and F1 as ideas. You will implement them in milestone 1, so you do not need a library.
- A laptop with 8 GB RAM. Milestone 4 downloads a 500 MB spaCy model and runs on CPU; the local LLM step there is optional.
- Read the AI Security kit's data-protection volume or know the DPDP Act, 2023 at the level of what a Data Fiduciary owes. Milestone 7 maps your controls to it.
- A GitHub account. The repository can stay private; you submit a link to it at the end.
Curriculum
3 sections · 9 parts · 1h 8m18 min free to read · 50 min with the project
01Before you start1 part · 6 min
- Before you startPreview6 min
02Milestones7 parts · 57 min
- Build the test bedPreview12 min
- Structured identifiers8 min
- Context and confidence7 min
- Names, with a model7 min
- The firewall8 min
- The evasion set7 min
- Cost, latency and the DPDP mapping8 min
03Wrap-up1 part · 5 min
- Wrap-up5 min
Twenty minutes that decide whether the next ten hours are a build or a struggle. Answer the six questions below honestly, check what you will have at the end against what you want, set up the folder, and read how this guide holds back on purpose.
Readiness check
Answer each one in your head, then open the answer. Score yourself: five or six right, start now. Three or four, start, but open the linked kit when the milestone named next to the question arrives. Two or fewer, read the linked kit first; this project assumes those ideas and does not teach them.
1. A detector flags 100 strings as Aadhaar numbers. 90 really are; it missed 30 others. What are its precision and recall, and which one would a privacy officer complain about first? (milestone 1, AI Security kit)
Submit your build
When the last milestone runs, hand in the repository and three lines about it. That is what the project page will show for you, and what you can send with a job application.
Sign in to submit your build: the repository link and three lines about it.
Reviews
to review this project once you have finished it.