CourseAI System Design: The Complete Course · Module 8: Orchestration and State · part 55 of 107
Part 55 · Module 8: Orchestration and State

Lesson 2: Sessions, context budgets and a cache that survives

31 min read·
Module 8 · Lesson 2 · About 55 minutes · Needs: Module 3 lessons 2 and 6 (history replay, prefix cache). One block calls a hosted model

The situation

Week 8. Joel wants follow-ups to work. Funmi asks what a long session does to the 1,500 ms first-token target, Ana what turn 30 costs, Leila when transcripts are deleted, Gareth what happens to a fact from turn 3 when turn 12 needs it.

The running example: Tallis employee t-tallis-u017 (finance) in a twelve-turn session on 5 October 2026, scripted for this lesson. Turn 3 says "My parental leave starts on 1 March 2027"; turn 12 asks "So when does my full pay end?"

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Or open everything with a plan