CourseLarge Language Models: The Complete Course · Module 3: Inference and Decoding · part 25 of 103
Part 25 · Module 3: Inference and Decoding

Lab: Put decoding settings in the package

31 min read·13 Oct 2026
Module 3 · Lab · About 60 minutes · Needs: lessons 1 to 3 of this module, and the Module 1 lab (or the cells in Step 1)

What you are building

SUPPORTDESK/DECODING.PY AFTER MODULE 3 generate() filters, seed local models DECODING one preset per task model_for() stream_timed() timings() TTFT, total checked() finish reason refuse length thinker() reasoning effort thinking tokens tests/ 9 tests no key needed imports Module 1: llm.py (ask, MODELS) and data.py (T-1001)
One new file (blue) holds every decoding decision, checked by nine keyless tests (green) and built on Module 1's files (grey).

It is Friday 19 June 2026, week 3. This week you timed replies, turned every sampling dial and paid for thinking, all in notebook cells. Maya's question was simple: which settings does each task use, and who decides? This lab answers it in one file, supportdesk/decoding.py, that every later module imports. It holds Lesson 2's sampling loop and per-task presets, Lesson 1's streaming timer, a finish-reason check that refuses unfinished replies, and Lesson 3's effort setting. Tests check all of it with no key and no model download. The running example is T-1001, Ana Duarte's duplicate charge. Only four calls go to the hosted model, all through the course's guard.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.