The Project: Northdesk

Every lesson in this course is set at one company, on one project. Northdesk is made up. The company, its people, its tenants, every document, request, trace and number in the project are fictional and simulated. The real cases that open each lesson are separate, cited, and clearly marked as real.

The company

Northdesk is a software company in Manchester. It sells Northdesk Workplace, an AI helpdesk for employees, to other companies, which it calls tenants. A tenant's employees ask about leave, expenses, laptops and pay; they ask for things to be done; and the tenant's HR, IT and facilities teams get fewer tickets. Northdesk has 240 tenants and covers 410,000 employees (seats). It charges $8 a seat a month for Ask and $4 more for Cases, in US dollars, the same currency the model providers publish prices in.

The product

The problem

Northdesk's first version was built fast, by a small team, for small tenants. Then larger tenants signed. On 14 September 2026 Tallis Retail Group switched Ask on for all 54,000 of its staff, and traffic in the sample doubled in two weeks. On 22 September the main model provider slowed down for 40 minutes and Ask timed out for some users. On 29 September a prompt change made answers longer; nobody connected it to the cost rise until Ana asked why the bill had jumped.

Joel, the head of product, has a target for the second quarter of 2027: 100,000 daily Ask users, 10 questions each, 2,000 input and 500 output tokens per question, a 3-second response target, and 60,000 Cases runs a day. Helen, the CTO, wants a design for the next version of the platform that meets it, with the evidence, before anyone writes production code.

You are the AI platform engineer Northdesk hired to produce that design and prove each decision. The target is the roadmap's worked estimate, so you will find out in Module 3 that the 3-second target cannot be met as written.

What this course uses the project for

This course teaches AI system design: what to put where, what to leave out, and which tradeoffs you are choosing deliberately. Northdesk is the instrument. Each module adds one decision and its proof to one repository, so by the capstone you can defend the whole design in a review, with every number regenerated from code.

It is not BrightCart (AI Engineering), Harbourline Freight (Forward Deployed Engineer), the returns desk (Agentic AI Engineer), the Quillpath support desk (Large Language Models), Hollin Answers (RAG), the notes assistant (MCP), the TinyFormer lab (Transformers), Booknook (Python) or the Tidewell Cycles inbox (NLP). Northdesk is a different company, with a different cast, data and problem.

The people

Tenant staff appear only as generated users in the shared data. Do not add named Northdesk staff; a tenant user may appear in one lesson only. The exceptions below were written before the consistency pass (23 October 2026) and are kept rather than renamed, because their lessons' code and the labs that copy it use them; they are part of the cast from now on. Golden-set requesters cited only by user id (for example t-tallis-u035) may recur.

The tenants in the data

Six of the 240 tenants have generated data. They were chosen to differ in every way that changes a design.

Each tenant's documents state different policy values (leave days, meal limits, pay day, salary bands), so an answer that quotes another tenant's value is a visible cross-tenant leak.

The rules in the documents

Every tenant has the same ten policy documents with its own values. These are what the golden questions ask about:

The values are in FACTS in 06-shared-data.md and never vary.

The Cases tool surface

Some requests must never be handled by the agent at all: a request to change bank details is escalated to a person with no tool called. The tools run as MCP servers in Module 9; in production each would sit in front of the tenant's real HR, asset and ticketing systems.

Dates

• The shared data covers Monday 7 September to Sunday 4 October 2026 (28 days). The snapshot was taken on Monday 5 October 2026 (DATA_AS_OF). Code counts days to DATA_AS_OF, never to date.today(), so lessons give the same answers whenever you run them.
• Traffic and trace times are in UTC; tenants' working hours follow their own time zones, which is why the busiest hour is when the UK afternoon overlaps the US morning.
• Dates in prose are written day first ("22 September 2026"); dates in data are ISO (2026-09-22).

Key events in the data

The two providers in the data are called primary and secondary on purpose. The incident is simulated and is not about any real company. In the lessons, primary is whichever provider you set in PROVIDER and secondary is the other.

The project weeks

The "design step" column refers to the roadmap's ten-step order in 01-how-this-course-works.md. Gemini 3.8 Flash's price doubles on 1 January 2027, in week 9: the price table in course-guide/02 says so, and the cost lessons use it as the example of a dated price.

Fixed names

So that modules fit together, these names do not change:

• The repository is northdesk-platform/; the package is northdesk/; tests live in tests/; evaluation in evals/; the web service in service/; design documents in docs/.
• northdesk/data.py holds blocks 1 to 6 of the shared data, copied exactly.
• northdesk/llm.py holds the setup cell from course-guide/02 and ask(model, messages), created once (Module 4 lab).
• Every component that calls a model goes through the model port, ModelPort.complete(messages, tier) -> Completion in northdesk/ports.py (Module 4), so the gateway (Module 7) and the scripted stand-ins used in tests are interchangeable.
• The gateway's public call is Gateway.complete(request) -> Completion, where the request carries tenant_id, feature, tier and budget_key, and the completion carries the text, the provider used, token counts, cost in US dollars and latency.
• Retrieval returns list[Hit] (document id, chunk id, score, text), already filtered by the user's groups and tenant: nothing downstream sees a chunk the user may not read.
• Every Ask answer is an Answer (text, cited document ids, status answered, not_found or provider_error).
• Every span from Module 9 on has trace_id, span_id, parent_id, name, start_ms, duration_ms and attrs, the same shape as TRACES.
• docs/numbers.md is generated by northdesk/estimate.py and never edited by hand.

One repository: which module adds which file

In Colab each lesson writes the files it needs with %%writefile and imports them; on a laptop the same files form the repository. Tests run with !python -m pytest -q in Colab and pytest -q locally. Every lab from Module 2 on starts by checking that the earlier modules' files are present and writing back any that are missing (the %%ensure cells), so you can start at any module.

The table lists every file a lesson or lab writes, generated on 23 October 2026 from the %%writefile and %%ensure cells in the published lessons and labs (each file under the module that first writes it; package __init__.py files are left out; "(generated)" marks a file written by code rather than by a cell). Lessons write their own small component files; the lab then writes the module's consolidated files, such as estimate.py, retrieve.py, model_registry.py or reliability.py. Module 7 Lesson 1's draft docs/adr/ADR-002-model-per-task.md is a draft number: ADR-002 is Module 5's data path, and the Module 7 lab records the model decision as ADR-004.

The design artifacts

Design documents are part of the deliverable, not decoration. Each is written from measured numbers.

The ADR template, used from Module 4 on:

# ADR-NNN: <decision in a few words>

Status: proposed | accepted | superseded by ADR-MMM
Date: <ISO date>
Deciders: <names from the cast>

## Context
<the problem, with the numbers from docs/numbers.md that force a decision>

## Options considered
<two or three, each with cost, latency, quality and what it locks in>

## Decision
<the choice, in one or two sentences>

## Consequences
<what gets easier, what gets harder, what we now have to operate>

## Revisit when
<the measurable condition that would change the decision, for example "Ask traffic passes 200 RPS at peak">

The finished platform

• A requirements document with an explicit error rate, cost asymmetry and refusal rules, as data the gates read
• A numbers sheet generated by an estimator that reproduces the roadmap's worked estimate and then Northdesk's own
• An ingestion pipeline that is incremental, idempotent, redacts personal data and keeps lineage
• Permissioned, tenant-isolated retrieval, scored for recall and for leaks
• A model gateway with routing, a cascade, a fallback chain, caching, budgets, a circuit breaker and rate limits, tested with fault injection
• A LangGraph Ask workflow with session state and compaction, and a job queue for the background paths
• A bounded Cases agent over MCP, with ceilings, approvals, a sandbox and an audit trail
• A FastAPI service tested with TestClient, a Docker Compose stack and a CI pipeline with eval gates
• An eval harness with gates, and traces in OpenTelemetry
• ADRs, data and request path figures, a trust boundary map, a failure-mode table and a migration plan

The one idea

Choose the boring version until the numbers say otherwise. Most proposed AI architectures are more autonomous, more multi-agent and more fine-tuned than the problem needs. Every component in this course has to earn its place with a number: a cost it saves, a latency it removes, a failure it contains, a leak it prevents.

Facts settled in lessons (18 October 2026, updated 23 October 2026)

Later lessons and labs use these values; they were added in the lessons named and are labelled there. Where a lesson's code still follows an earlier reading, its prose says so and the module lab applies the value below.

• Model spend may be at most 10% of seat revenue: with the $8 Ask seat and about 220 requests per seat a month, the cost ceiling is $0.003636 per Ask request (8 x 10% / 220, written to six decimal places; earlier text rounds it to $0.0036) (Module 2 Lesson 3).
• 22 working days a month; Tallis's staff-chat webhook waits 30 seconds; the team has no GPU operations experience (Module 2 Lesson 3).
• Residency, read strictly from provider pages checked 18 October 2026 (residency.py, Module 2 Lesson 3; this code wins where prose differs): the Gemini Developer API (gemini-paid) cannot be shown to keep processing in any named region, so it fails residency for every tenant, US included; Groq is "ask" and passes only with data retention switched off, pending Leila's check; managed cloud regions are "ask" in residency.py and are judged per model and region by northdesk/region_check.py (Module 10 Lesson 4). Designs that send a tenant's data to a model must route it to a provider or region that passes, or say why not (Module 2 Lesson 3).
• Illustrative error prices: $4 unanswered, $40 wrong, $2,000 unsafe; human review capacity 400 answers a day (Module 2 Lesson 2).
• Caching on Gemini: Google's caching page (checked 18 October 2026) gives a 4,096-token minimum for Gemini 3.8 Flash, so an Ask prompt of about 2,000 tokens cannot be cached there; guide 06's cached Ask traces only make sense on Groq. Gemini's cached input price also doubles on 1 January 2027 ($0.075 to $0.15 per million), and cache storage costs $0.50 per million tokens per hour (Module 3 Lesson 4).
• Self-hosting break-even uses Module 3 Lesson 3's sizing (2.65 GPUs of work, 4 GPUs, $5,840 a month) and illustrative people costs of $3,000 or $10,000 a month (Module 3 Lesson 4).
• First-token target: Ask's time to first visible token is at most 1,500 ms at p95 (proposed in Module 3 Lesson 5; Joel's 3-second target becomes this once answers stream).
• Freshness targets: policy documents searchable within 1 hour of a change, everyday documents within 24 hours, both at p95; uploads limited to 50 MB; Marlow edits its leave policy on 5 October (carry-over five to eight days, no FACTS value changes) (Module 5 Lesson 1).
• Retention proposal (not legal advice): answer cache 1 day, backups 29 days, traces 30 days (Module 5 Lesson 2). MinIO's repository was archived on 25 April 2026; the course's self-hosted object store example is SeaweedFS (Apache 2.0). Qdrant local mode ignores payload indexes.
• 22 September fallback, corrected 19 October 2026: because the Gemini Developer API fails residency for every tenant (residency.py), no tenant may fall back to it. Fallback goes to Groq (retention off, "ask" pending Leila) or to degraded mode. Module 7 Lesson 2's replay, which let US tenants fall back to Gemini, and guide 06's traces, which record 27 fallbacks to the secondary including 16 for UK and EU tenants, show the first design breaking this rule; later lessons teach that as the mistake the rule fixes.
• Bedrock residency (Module 7 Lesson 3, checked 18 October 2026): Bedrock's gpt-oss-20b model card lists In-Region availability in eu-west-2 (London), which keeps requests within one region, so it can pass UK residency. Google's managed gpt-oss-20b (gpt-oss-20b-maas) is us-central1 only and is to be retired on 21 October 2026.
• Retrieval (Module 6 Lesson 1): retrieval budget 280 ms (from Module 3 Lesson 5), measured 69.8 ms for the whole pipeline on one CPU; chunks are paragraphs capped at 180 words with the document title in front.
• Memory retention proposal (not legal advice): open case 30 days from last use, preference 180 days, reviewed team and organisation notes 365 days; memory adds a 150-token budget per request (Module 8 Lesson 3).
• Sessions (Module 8 Lesson 2): context budget 4,000 tokens per Ask request (latency bound 5,394), history allowance 1,105 tokens, 30-minute idle timeout (design assumption).
• Cloud Run GPUs (Module 10 Lesson 3, checked 18 October 2026): the GPU page lists no London region; L4 GPUs are in Belgium, the Netherlands, Iowa, Northern Virginia, Singapore and Mumbai (invitation only). One GPU serves about 17.5 RPS (from Module 3 Lesson 3).
• Groq retention (Module 10 Lesson 5): with data retention on, Groq stores data in the US and fails UK and EU residency, so Northdesk runs Groq with retention switched off. HashiCorp Vault is Business Source License from 1.15.0; OpenBao (MPL-2.0) is the open-source option.
• Release bundle names (Module 10 Lesson 6): the 29 September change is versioned as prompt ask-v8 plus config cfg-12 (12 chunks) on an unchanged model; v7's bundle is prompt ask-v7 with cfg-8. A canary would have rolled it back at 10:50 after 205 requests (output tokens +40%); with no canary 221,334 requests got v8.
• Residency by cloud (Module 10 Lesson 4, pages checked 18 October 2026; not legal advice): the managed clouds now state where processing happens, so a model-and-region pair can pass. For UK tenants each cloud has one passing option: gemini-3.5-flash on Gemini Enterprise Agent Platform in europe-west2 (the Gemini Developer API still fails), gpt-oss-20b on Bedrock In-Region in eu-west-2, and gpt-4.1-mini on Foundry Standard in uksouth. Google's quickstart 'global' location fails. Bedrock EU Geo routing and Azure EU Data Zone stay 'ask' for EU tenants. residency.py still says 'ask' for managed regions; northdesk/region_check.py (place_ok over model_regions.py) gives the per-pair answer.
• Semantic cache (Module 10 Lesson 1): threshold 0.95 per tenant scope (local MiniLM put different questions above true paraphrases); cache keys always include the tenant.
• Practice-design ADRs (Module 15 Lessons 3 and 4) go in docs/practice/, not docs/adr/: they are exercises, not Northdesk decisions, and stay out of the decision register.
• Two numbers settled by the Module 15 lab (northdesk/review/settle.py): the cost ceiling is $0.003636 everywhere, and CONSTRAINTS["latency"] in northdesk/requirements.py says first token within 1.5 s at p95, replacing Module 2's 3.0 s, to match ADR-000.
• Planted notes (Module 5 lab, Module 12 lab): Module 5's Intake quarantines guide 06's planted notes, so on the retrieval path they never reach a model. Module 12's lessons treat the pre-filter on its own, where the notes pass; the Module 12 lab found that the Module 10 service built Ask's candidates from raw documents, which let a planted note into context for 6 of 54 golden questions, and service/trusted.py (harden()) closes that path.
• Cases tool names: guide 03's eight tools are the per-intent surface (Module 9 Lesson 2). The MCP name is hris.get_employee; direct function-calling definitions use hris_get_employee (dots become underscores, Module 9 Lesson 2), and Module 9 Lesson 3's tool gateway uses <server>__<tool>, hris__get_employee.