Lesson 2: Feedback loops and what the frontier changes
35 min read·13 Oct 2026
Module 13 · Lesson 2 · About 60 minutes · Needs: Module 11 Lesson 5 (error analysis, user signals, regression gates) and Module 12 Lesson 2 (prices and prompt caching)
The situation
From the pilot on 2 November, Hollin Answers will log thousands of queries a week. Grace wants a weekly routine: find what went wrong, fix the worst first, prove nothing else broke. Owen asks why not hand a long-context model every document and drop retrieval.
It is Friday 9 October 2026. The running example is one logged query: "x300 motor warranty how long", typed by an agent at 09:14 on 21 September. The Hollin data is simulated.