CourseLarge Language Models: The Complete Course · Module 3: Inference and Decoding · part 22 of 103

Part 22 · Module 3: Inference and Decoding

Lesson 1: How a model writes its reply, one token at a time

23 min read·13 Oct 2026

Module 3 · Lesson 1 · About 55 minutes · Needs: Module 1 Lesson 1 (TinyLM and the generation loop) and Module 2 Lesson 1 (tokens)

The situation

It is Thursday 18 June 2026, week 3. You show Maya and Lena a drafted reply to T-1001, Ana Duarte's duplicate charge. Lena watches a blank pane for a few seconds, then the whole draft appears at once. "Can it show the words as they come? And why does a long draft take so much longer, when the prompt is the big part?"

Both questions are about how a model produces a response. This lesson opens the loop that writes every reply and times each part of it, following Ana's ticket throughout.

Today Lena sees nothing (grey: waiting) until the model (amber) has written the last token, then the whole T-1001 draft appears at once.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399 See what is inside