How This Course Works
One platform, designed and proved piece by piece
You do not study fifteen separate subjects. You design one system, the next version of the platform behind Northdesk, a made-up company that sells an AI helpdesk to other companies, and every module adds one decision to it, with the evidence for that decision. Some evidence is arithmetic, some is a small component that runs in a notebook, some is a failure you cause on purpose and then handle. By the end you have one repository: a design document, the numbers behind it, the decisions and why you made them, and runnable code that proves the design works.
The project, its people, its data and its dates are described in 03-northdesk.md. It is fictional; the real cases that open each lesson are not.
Who it is for
You can write Python, you have called a language model from code, and you know what an API, a database and a queue are. You may be an engineer preparing for an AI system design interview, an engineer who has built AI features and now has to decide how a whole system fits together, or a lead who reviews other people's designs. If you have never called a model, start with the AI Engineering course in this series; if you want depth on one layer (retrieval, agents, MCP, serving), the module READMEs point to the course that owns it.
Why the numbers come before the architecture
Most AI design mistakes are visible in the arithmetic five minutes in. The course follows the order the roadmap recommends for any design, on a whiteboard or in a document:
- Clarify the problem (Module 2).
- Set the quality bar (Module 2).
- Run the numbers (Module 3).
- Pick the simplest architecture the numbers permit (Module 4).
- Draw the data path (Modules 5 and 6).
- Draw the request path (Modules 7, 8, 9 and 10).
- Mark the trust boundaries (Module 12).
- Say how you would know it works (Module 13).
- Name the failure modes (Module 11).
- State the tradeoffs, and what you would revisit at ten times the scale (Modules 14 and 15).
Module 1 explains why an AI system needs this order, and Module 15 walks it end to end on Northdesk.
Pictures first
Every lesson is a walk through one example, one picture at a time. A figure shows an idea; a short paragraph says what the figure shows in plain words; a short block of code proves it; a walkthrough explains every line of that code. Then the next figure. A lesson has fifteen to thirty figures and takes thirty to forty-five minutes to read properly.
The same colours mean the same things in every figure:
| Colour | Means |
|---|---|
| Blue | Your system, your code, the normal request path |
| Amber | A model, or anything uncertain (a model's answer, a score, untrusted input) |
| Green | Correct, safe, passed, within budget |
| Red | Failure, a leak, danger, over budget |
| Grey | Optional, background, a managed service you do not run |
Open source first, managed and closed shown beside it
Every mechanism is built the plain way first (a router in plain Python before a gateway library, a span log before OpenTelemetry), then with the open-source framework that does it properly (LangGraph, the MCP SDK, FastAPI, RQ, Qdrant, Phoenix). The managed or closed options a real team might choose instead (Amazon Bedrock, Google's Gemini Enterprise Agent Platform, Microsoft Foundry, hosted gateways, managed vector stores, hosted tracing) are shown side by side in a comparison figure, with the official code snippet labelled not run here. You learn what each side costs, what you operate, and what you are locked into, so you can choose either with reasons.
The shape of every lesson
| Part | What happens | Why it is there |
|---|---|---|
| The situation | A decision Northdesk has to make, with named people | You learn the idea because you need it |
| A real case, first | Something that really happened, from an engineering blog, incident write-up, paper or talk, cited | So you know the idea matters outside a course |
| The idea | Figures and short paragraphs, one running example throughout | The picture comes before the vocabulary and the formula |
| Run it | A short piece of code, pasted into Colab, with a code walkthrough | You see the claim proved, and you understand every line |
| Break it | You make it fail on purpose | Real systems fail; see the failure here first |
| In production | What teams do at scale, including the managed options | The part most courses leave out |
| The question a reviewer will ask | One question in the voice of Gareth, Northdesk's design reviewer | Interview and design-review practice |
| Exercise | One task with a runnable check | You do it, not just read it |
| Key terms and sources | A short glossary and the sources the lesson used | So you can go further |
Three depths of topic
Each topic is marked in its heading:
| Mark | Meaning | What you get |
|---|---|---|
| (Essential) | You cannot design the system without it | Figures, code or a measurement, and a "break it" step |
| (Recommended) | Skip it only if you have a reason | A figure and a short section, often with code |
| (Optional) | Knowing it exists is enough | An "awareness card" |
Some topics appear in more than one place in the roadmap (for example "Durable execution and checkpointing" in both orchestration and agents). Each is taught once. Where it comes back, you get a recall card: one figure, one paragraph, a link to the lesson that teaches it, then only what is new.
What every module contains
| File | What it is |
|---|---|
README.md | What the module is for, what you need first, which step of the design order it covers, the lesson list |
| Numbered lessons | One file per group of topics |
...-module-lab.md | A guided build that adds this module's decision and its proof to the Northdesk repository |
...-incident-file.md | A real, published failure, then the same failure replayed on Northdesk and designed out |
Module 16 is the capstone: the Northdesk design review. You present the whole design in the ten-step order, with every number regenerated and every component passing in one run, then take two practice designs to a full design document.
Skip what you know
Each module README starts with What you need first, so you can check whether it is safe to jump ahead. If you are preparing for an interview, Modules 3 (the arithmetic), 4 (choosing an architecture) and 15 (communicating a design) are the ones not to skip.
How to run the code
All code runs in Google Colab on the free CPU. Open a new notebook, paste each code block into a cell in order, and run it. No lesson needs a GPU: the GPU topics are arithmetic, a timing on a small model, and serving commands that are given but not run.
About half the lessons call a hosted language model. They start with the same short setup cell, which lets you choose between three providers by changing one word: Groq and Gemini work in Colab with a free key; Ollama runs a model on your own computer. Setup, the pinned libraries and the one table of model prices: 02-setup-colab-and-llm-providers.md. Running the whole repository on your own computer, with Docker Compose for the parts Colab cannot run: 07-run-locally.md.
Some things cannot run in a notebook: containers, Kubernetes, GPU serving, real queues and databases as servers, cloud services, CI pipelines. Those sections give exact instructions from the official documentation, say plainly that they were not run in the course, and say what was checked instead.
The data is generated, not downloaded
All Northdesk data is made by code with a fixed seed: six tenants, 360 users, 471 permissioned documents, 28 days of traffic, 1,560 traces with the latency and token count of every hop, a dated price table and two labelled golden sets. Every learner has exactly the same data, and nothing has to be downloaded. Lessons paste only the blocks they need from 06-shared-data.md and print a short fingerprint you can compare with the one on that page. All of it is simulated, and lessons say so.
Every output is real
Every output printed in a lesson came from running that lesson's code, on the date shown. There are no hand-written "example outputs". Where a lesson tests plumbing (a fallback, a circuit breaker, a retry) with a scripted stand-in instead of a model, it says so, and the stand-in's replies are never presented as what a model said. Numbers in the text come from those runs, from the shared data, or from a cited source. Prices are a dated snapshot and say so.
Your numbers may differ a little
The arithmetic and the simulations are exact: the same cell on the same data gives the same numbers. Two things differ from run to run. Hosted models do not return the same words or the same token counts every time, even at temperature 0, and some ignore the temperature setting altogether. And measured latency depends on the provider's load at that moment, so your time to first token will not match the lesson's. Module 3 lesson 5 shows how to measure it properly, with percentiles rather than one number.