Lesson 1: Serving Ask fast, cheaply and safely
34 min read·
Module 10 · Lesson 1 · About 60 minutes · Needs: Module 3 lessons 1, 5 and 6 (concurrency, the first-token budget, the prefix cache), Module 6 lesson 1 (embeddings). One optional block calls a hosted model; the rest uses scripted stand-ins
The situation
Week 10, 4 January 2027. Ask has a budget: first visible token within 1,500 ms at p95, retrieval allowed 280 ms (Module 3 lesson 5). Now it needs a service that keeps it. Funmi asks what happens when Tallis's staff all ask at 9 o'clock; Ana, which questions can skip the model; Leila, who may share a cached answer.
The running example: Vera Park (t-tallis-u004, Tallis operations) asks "How many days of annual leave do I get?" Tallis's 2026 policy says 30.