CourseAI System Design: The Complete Course · Module 3: Capacity and Cost Estimation · part 25 of 107
Part 25 · Module 3: Capacity and Cost Estimation

Lesson 6: Cost decisions you build in, not bolt on

24 min read·
Module 3 · Lesson 6 · About 50 minutes · Needs: Module 3 lesson 1 and the worked estimate's cost line; enough Python to run a cell. No model calls and no key.

The situation

Week 3 at Northdesk. At Joel's target, Ask sends 2,000 input and 500 output tokens a million times a day; the roadmap prices that month at about $135,000 with illustrative prices. Ana asks which parts of the bill the design fixes. Funmi asks what stops one leaked key spending the month in an afternoon.

Follow one question through the lesson: an Oriel Software employee asks, "How many days of annual leave do I get?"

Six choices, six places Ask's request path and the paths beside it Active flow Background or async 1 Processing step number Cost controls router tiering budget quota ceiling stop Cache answer cache LLM Generation Layer model prefix cache 1 2 3 4 Off the request path, nobody waits nightly job precomputed answers batch path Intake, Digest
Each of the six choices has a fixed place: router, budget, ceiling and caches on Ask's path (the model in blue), precomputation and batch beside it (dashed), so each must be designed in.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan