CourseNatural Language Processing: The Complete Course · Module 10: LLMs for NLP, and Responsible NLP · part 70 of 73
Part 70 · Module 10: LLMs for NLP, and Responsible NLP

Lab: The bake-off, every model on one test set

114 min read·14 Oct 2026
Module 10 · Lab · About 75 minutes · Needs: lessons 1 to 4 of this module, and the Module 9 lab (or its cells, pasted here). CPU only; a Groq or Gemini key only if you switch the hosted rows on

What you are building

HARD_TEST 80 messages 4 local models 2 hosted LLMs off by default fakes check code score same metric cost router name swap one report: reports/bakeoff.json
Local models (blue) and hosted LLMs (amber, off until you add a key) meet the same 80 messages and the same score; cost, router and name swap follow (green).

It is Thursday 12 November 2026, project week 10. Rosa wants one table before she decides anything: every model the course has built, on the same 80 hand-written messages, with what each costs and whether a customer's name changes the answer.

This lab settles the four lessons' files once, then runs the bake-off: four local models for real, two hosted LLM rows switched off until you add a key, a cost table from course-guide/02's prices, a hybrid router and a name-swap check. It ends with one report in reports/.

Open a new Colab notebook, call it module-10-lab.ipynb, and run each cell in order.

The rest of this part opens with an account

Signing up is free and takes one email. Nothing to pay, and nothing renews.