Lesson 1: Latency budgets, parallel stages and the slow tail

Module 12 · Lesson 1 · About 45 minutes · Needs: Module 1 Lesson 2 (the stage trace), Module 6 Lesson 2 (fusion), Module 8 Lesson 1 (the reranker) and Module 9 Lesson 2 (streaming)

The situation

The agent pilot starts on Monday 2 November 2026. Grace has watched agents use Kwame's prototype: if nothing appears for a couple of seconds, they search the help centre by hand instead. Her target: the first words of an answer on screen within 1.5 seconds, for 95 questions in every 100.

Kwame asks whether the index will hold that when every agent asks at once. This lesson times Ellie's question stage by stage, splits the 1.5 seconds, and tests the index under load. Hollin and its targets are simulated.