CourseAI System Design: The Complete Course · Module 11: Reliability · part 81 of 107
Part 81 · Module 11: Reliability

Lab: Gateway version 3 and the 22 September replay

353 min read·
Module 11 · Lab · About 70 minutes · Needs: lessons 1 and 2 of this module, and the Module 10 lab (or nothing: Step 1 rebuilds Modules 1 to 10)

What you are building

Gateway version 3 Mechanisms in front, a ladder behind Active flow 1 Ladder rung, in order Optional or fallback Gateway version 3 rate limits tenant, person bulkhead per feature Guarded 4 s a try, 8 s LLM Generation Layer groq primary retry budget and circuit breaker inside Guarded; residency from model_registry.ALLOWED When the primary fails, the ladder, in order the same rules wrap Module 10's service: GuardedStream inside, limits in front Gemini API fails residency 1 fixed answer from code 2 documents provider_error 3 static message provider_error 4
Gateway version 3: Northdesk's mechanisms (teal) in front of the primary (grey, not run by Northdesk). The Gemini API fails residency for every region (pale red), so after the primary the ladder serves a fixed answer, then the documents or a static message, both still provider_error (dashed fallbacks).

This lab joins the lessons' parts into gateway version 3, northdesk/reliability.py: per-tenant and per-person rate limits, a bulkhead per feature, and Module 7's gateway over backends that each have a timeout, a share of one retry budget and a circuit breaker. Lesson 1's ladder runs after it, and the same rules wrap Module 10's service.

Then you replay 22 September from 13:00 to 13:40 UTC under the corrected residency rule: residency.py fails the Gemini API in every region, so no tenant has a secondary. Last, every row of docs/failure-modes.md gets a fail mode and a test, or a reason why not.

No model is called: providers are scripted stand-ins, never model output. Open a new Colab notebook, module-11-lab.ipynb, and run each cell in order.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan