CourseAI System Design: The Complete Course · Module 10: Deployment and Serving · part 76 of 107
Part 76 · Module 10: Deployment and Serving

Lab: The Northdesk service, packaged and gated

317 min read·
Module 10 · Lab · About 75 minutes · Needs: lessons 1 to 6 of this module, and the Module 9 lab (or nothing: Step 1 rebuilds Modules 1 to 9)

What you are building

One service, packaged and gated The lab's finished piece Active flow Background or async 1 Processing step number Client Vera, Maya a token each API Layer FastAPI service/app.py /ask /cases /jobs /healthz /readyz LLM Generation Layer model stream answers /ask Worker Tiers RQ RQ tiers 3 workers 1 2 3 Around it cache and gate slot freed always release bundle v8, v9 on 5% Packaging Docker, Compose, CI written, not run
The lab's piece: one service (green) in front of the model's stream (blue) and three worker tiers (yellow). The cache and gate (teal) protect the model, a release bundle decides what each seat is served, and the packaging files are written but not run here.

The lessons built the serving pieces one at a time. This lab joins them into one FastAPI service, service/app.py, that a Dockerfile starts, Docker Compose runs beside its workers and stores, and GitHub Actions gates. On the way it fixes a bug Lesson 1 left in: a client that leaves before its stream starts keeps its admission slot for ever.

Every number comes from scripted models, fakeredis and TestClient; one optional cell calls your provider. Open a new Colab notebook, module-10-lab.ipynb, and run each cell in order.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan