Setup: Colab, Libraries, Model Providers, and Running Locally

This takes about ten minutes. You do it once.

Most of this course needs no key at all. Rules, tokenisers, spaCy, scikit-learn, gensim, sentence-transformers and the Hugging Face models all run inside the notebook. A hosted language model appears in Module 10, and in a handful of optional or recommended comparisons before it:

Modules 2 to 7 make no hosted calls. If you only want the classical and Hugging Face parts of the course, you can skip sections 2 to 6 of this page until Module 10.

1. Colab, CPU and GPU

All code runs in Google Colab. Open a new notebook, paste each code block into a cell in order, and run it.

Every lesson runs on Colab's free CPU except Module 7 Lesson 4 and the Module 7 lab, which fine-tune DistilBERT. A T4 GPU makes that much faster: choose Runtime, Change runtime type, T4 GPU before you start. Those lessons also give a CPU path that trains on a subset of the data for one epoch with max_length=64, so you can finish without a GPU. Training times quoted in the course name the hardware they were measured on.

The course targets the Python version Colab runs, which was 3.12 at the last check recorded for this series (13 October 2026). Print it once in your first notebook:

import sys
print(sys.version)

2. Choose a model provider (Module 10, and the optional calls)

The course works with three ways of reaching a language model. You only need one. You switch by changing one word in the setup cell.

If you are using Colab, choose Groq or Gemini. They need a free key and nothing else. If you want Ollama, work on your own computer (section 9). Ollama runs the model on your machine, so no key is needed and no message leaves your computer, which matters for Amara's question about customer text.

Free plans limit how many requests you can send per minute and per day. If you hit a limit you get an error with the number 429. Wait a minute and try again, or switch provider.

Model names change. The names below come from the series' price table, checked against each provider's documentation on 9 October 2026, and were listed again on the providers' model pages on 14 October 2026. This course was planned without fresh web checks after that day. If a call fails with "model not found", open the provider's model list (links below) and put a current name into the setup cell. Nothing else in the lesson needs to change.

3. Get a key (Groq or Gemini)

Groq

1. Go to console.groq.com and sign in.
2. Open API Keys and create a key.
3. Copy it. You will not be shown it again.

Model list: console.groq.com/docs/models

Google Gemini

1. Go to aistudio.google.com and sign in.
2. Choose Get API key and create a key.
3. Copy it.

Model list: ai.google.dev/gemini-api/docs/models

4. Store the key safely in Colab

Never paste a key into a code cell. Notebooks get shared, and a key in a shared notebook is a key someone else can spend.

1. In the left sidebar of your notebook, click the key icon (Secrets).
2. Add a secret. The name must be exactly GROQ_API_KEY (for Groq) or GOOGLE_API_KEY (for Gemini). The value is your key.
3. Turn on Notebook access for it.

The secret is remembered across notebooks. For each new notebook you only switch access on.

5. The setup cell

Every lesson that calls a hosted model begins with these two cells. Paste them, set PROVIDER, and run.

The course uses LangChain, a widely used Python library for building applications on language models. One of the things it does is give every provider the same interface, so the rest of your code does not care which one you chose. In this course LangChain appears only in Module 10, after a plain-Python version of each idea.

%pip install -q -U langchain langgraph langchain-groq langchain-google-genai langchain-ollama

import os
from langchain.chat_models import init_chat_model

PROVIDER = "groq"   # "groq" or "gemini" in Colab; "ollama" on your own computer
MODELS = {"groq": "groq:openai/gpt-oss-20b", "gemini": "google_genai:gemini-3.5-flash-lite", "ollama": "ollama:llama3.2:3b"}
KEY = {"groq": "GROQ_API_KEY", "gemini": "GOOGLE_API_KEY", "ollama": None}[PROVIDER]
if KEY and KEY not in os.environ:
    from google.colab import userdata   # keys live in Colab's Secrets panel
    os.environ[KEY] = userdata.get(KEY)
llm = init_chat_model(MODELS[PROVIDER], temperature=0, max_retries=6)

print(llm.invoke("Reply with the single word: ready").text)

If it prints ready (or something close), you are set up.

Code walkthrough

• %pip install ... installs the libraries once per notebook. langchain is the framework (with langgraph, its companion for multi-step workflows, kept in the standard cell so that every course in the series uses the same one); the three langchain-... packages are the connectors for each provider.
• PROVIDER = "groq" is the one word you change to switch provider.
• MODELS names the model for each provider as provider:model-name. This is the only place model names appear, so when a name changes you edit one line.
• KEY is the name of the secret the provider needs. Ollama needs none.
• The if block runs only in Colab when the key is not already set: it copies your key from the Secrets panel into an environment variable, which is where LangChain looks for it.
• init_chat_model(...) creates the model object, llm. temperature=0 asks for the most consistent output. max_retries=6 makes the client wait and try again when the provider says "too many requests" (a 429); Groq's free plan allows 8,000 tokens a minute, which a batch of classification calls can pass. It does not retry a reply the provider rejected as malformed: see "When the provider rejects a reply" below.
• llm.invoke(...) sends one message and returns a reply; .text is the reply as a string.

Why one cell works for three providers

init_chat_model reads the part before the colon (groq, google_genai, ollama) and builds the right connector. All connectors offer the same methods: .invoke() to get a reply, .stream() to get it piece by piece, and so on. Your code talks to llm and never mentions the provider again. The provider is a setting, not something written all through the code, which is also how real teams avoid being locked to one provider.

The connectors are not identical underneath. One difference the series has met: with structured output, Groq applies strict=True and the Gemini connector ignores it without a warning (seen in the series' runs, October 2026), so Module 10 always validates the reply itself.

When the provider rejects a reply

Sometimes a provider refuses the model's own reply with an error instead of returning it. In the series' test runs on 12 October 2026, Groq did this a few times with openai/gpt-oss-20b: an answer that should have been JSON came back empty, or the model called a tool it was never given. max_retries does not help, because the request itself was fine. The course handles it the same way everywhere: try once more, and if that also fails, say so plainly instead of crashing.

def ask(model, messages):
    """Returns (reply, None), or (None, reason) after two refusals."""
    for attempt in range(2):                     # one retry: the next reply is usually fine
        try:
            return model.invoke(messages), None
        except Exception as err:
            last = err
    return None, f"provider_error: {type(last).__name__}"

Code walkthrough

• ask sends the messages and returns the reply with no reason, or no reply with a reason.
• On an error it tries once more; after a second error it gives up and names the error, for example provider_error: BadRequestError.

Lessons use this shape: a message the LLM could not label gets the result "provider_error" and is counted as wrong in the score, so one bad reply never stops a notebook and never quietly improves a result. Module 10 caches every reply to disk, so a rerun does not pay twice and stays inside Groq's daily limit.

Prices and limits (checked 9 October 2026)

This is the course's one table of model prices and limits, the same as in the other courses of the series. Lessons that count cost say "prices from course-guide/02" and use these numbers; when a provider changes them, this table changes and the lessons follow. Prices are US dollars per 1,000,000 tokens on each provider's paid plan.

Sources: Groq, "Supported models" (https://console.groq.com/docs/models) and "Rate limits" (https://console.groq.com/docs/rate-limits); Google, "Gemini Developer API pricing" (https://ai.google.dev/gemini-api/docs/pricing). Free plans change often and your own limits are on each provider's console page; these are the numbers in this course most likely to go out of date. On 14 October 2026 Groq's models page still listed llama-3.3-70b-versatile, which another course in the series recorded as shut down on 16 August 2026 from Groq's deprecations page: do not use it, and check the deprecations page when Module 10 is written.

Module 10 quotes cost per thousand messages, computed from this table and from token counts measured in the lesson. Tidewell's own money is in pounds; model prices stay in dollars, as the providers publish them.

6. Closed models, shown but not run

Lessons may show how the same call looks with OpenAI or Anthropic models, using the official SDK or LangChain snippet from their current documentation, labelled "not run here". The course does not call them, so no key is needed and no number in the course comes from them.

7. The NLP libraries

Colab already ships torch, transformers, scikit-learn, numpy and pandas, and usually spacy, nltk and gensim. Each lesson installs only what it imports, in one cell near the top, with the version pinned so that your run matches the course's. numpy, scipy, pandas and torch are left at Colab's own versions, because forcing them can make Colab ask you to restart the runtime; the lesson pins one of them only if it fails without the pin.

Versions are the latest releases on PyPI, checked on 14 October 2026; "Released" is the upload date of that release.

The huggingface-hub pin. The newest huggingface-hub was 2.2.0 (8 October 2026), but sentence-transformers 6.1.0 and langchain-huggingface 1.2.3 both require a version below 2.0. Colab may already have a 2.x version, and pip may also choose 2.x for another package. Every cell that installs sentence-transformers or langchain-huggingface therefore pins it explicitly, and the same pin is safe for transformers 5.19.0:

%pip install -q sentence-transformers==6.1.0 huggingface-hub==1.33.0

This is the pin most likely to change: when sentence-transformers releases a version that accepts huggingface-hub 2.x, this page changes and the lessons follow.

Checked on paper, not yet in Colab. On 14 October 2026 a pip install --dry-run of the whole table (with huggingface-hub==1.33.0), for Python 3.12 on Linux, wheels only, resolved with no conflict: 138 packages. That proves the pins fit together; the first lesson pilot installs and imports them in Colab and records the result here.

If pip reports a version conflict in Colab, keep the pinned library and let pip choose its dependencies. If Colab asks you to restart the runtime after an install, do so and run the cells again from the top.

Data the libraries download

Some libraries need extra files, downloaded in the cell that first needs them. None needs an account.

After a spaCy download in Colab, load the model in a new cell; if it says it cannot find the model, restart the runtime once and run the cells again. The lesson prints nlp.meta["version"] so you can check you have 3.8.0.

Not used, on purpose. The Spanish and French spaCy models (es_core_news_sm, GPL 3.0; fr_core_news_sm, LGPL-LR) are left out because of their licences; the multilingual lessons use multilingual sentence-transformers models instead. Models released for non-commercial use only (for example facebook/nllb-200-distilled-600M and facebook/fasttext-language-identification, both CC BY-NC 4.0) are not used either. fastText vectors are trained in the lesson with gensim rather than downloaded.

8. Open models from Hugging Face (no key)

These models are downloaded in the notebook and run there, with no key. An HF_TOKEN secret only removes Hugging Face's anonymous-download warning and rate limit; if you add one, name it exactly HF_TOKEN in Colab's Secrets panel. Licences were read from the Hugging Face Hub on 14 October 2026; lessons state them as recorded here and do not re-check them. Module 7 Lesson 3 shows how to pin a model to a fixed revision, so a changed upload cannot change your results; the revisions the lessons pinned are listed after the table.

Pinned revisions (full commit ids, recorded when the lessons were run on 14 October 2026):

The course uses no downloaded datasets: all Tidewell data is generated in code (06-shared-data.md).

9. Running on your own computer (required for Ollama)

You can run every lesson locally, with any provider. For Ollama it is the way to do it.

Install

1. Python 3.12 (3.13 also works) from python.org. Not 3.14 yet: gensim 4.4.0 has no wheels for it. Older than 3.12 will not work either, because the pinned numpy and scipy need 3.12. Check with python --version (on some systems python3 --version).
2. A code editor. VS Code with the Python and Jupyter extensions is a good choice. Cursor, PyCharm and others work the same way.
3. Ollama, if you are using it, from ollama.com/download. After installing, download the course's default model once, in a terminal:

ollama pull llama3.2:3b

That is a download of about 2 GB. The model runs acceptably on a recent laptop with 8 GB of memory or more. A small local model makes more mistakes than the hosted ones, and Module 10 measures how many.

Create the project folder

mkdir tidewell-inbox
cd tidewell-inbox
python -m venv .venv

Switch the virtual environment on:

# macOS and Linux
source .venv/bin/activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

Create requirements.txt with the course's pins (the Module 1 lab writes the same file):

regex==2026.9.29
ftfy==6.3.1
pytest==9.1.1
nltk==3.10.3
spacy==3.8.16
tokenizers==0.23.3
transformers==5.19.0
huggingface-hub==1.33.0
scikit-learn==1.9.1
gensim==4.4.0
sentence-transformers==6.1.0
rank-bm25==0.2.2
faiss-cpu==1.15.1
datasets==5.1.0
accelerate==1.15.0
torch==2.14.1
pydantic==2.14.1
sacrebleu==2.6.0
rouge-score==0.1.2
seqeval==1.2.2
bert-score==0.3.13
numpy==2.5.4
pandas==3.0.6
scipy==1.18.1
matplotlib==3.11.2
langchain==1.4.5
langchain-core==1.6.10
langgraph
langchain-groq==1.1.3
langchain-google-genai==4.4.2
langchain-ollama==1.1.0
langchain-huggingface==1.2.3
python-dotenv
ipykernel

Then install everything, and the language data, once:

pip install -r requirements.txt
python -m spacy download en_core_web_sm
python -m spacy download de_core_news_sm
python -c "import nltk; [nltk.download(p) for p in ['punkt_tab', 'stopwords', 'wordnet', 'vader_lexicon']]"

This is a large install (PyTorch alone is several hundred megabytes). The pins resolved together in a dry run on 14 October 2026; they have not yet been installed on every operating system, so if pip reports a conflict on yours, keep the pins of the libraries you are using in the current module and let pip choose the rest.

Keys on your own computer

Ollama needs no key. For Groq or Gemini, create a file named .env in the project folder:

GROQ_API_KEY=paste-your-key-here
GOOGLE_API_KEY=paste-your-key-here

and add these two lines at the very top of the setup cell:

from dotenv import load_dotenv
load_dotenv()

Never share the .env file or commit it to Git; the Module 1 lab puts it in .gitignore.

Run the lessons

Open the folder in your editor and create a notebook file, for example module-04-lesson-1.ipynb. Select the .venv Python when asked. A notebook in VS Code works like Colab: paste each code block into a cell and run it. Skip the %pip install cells, because you installed the libraries already, and set PROVIDER = "ollama" in the setup cell if you are using Ollama. Lines starting with ! (such as !pytest -q) work in a notebook; in a terminal, type the command without the !.

On a Mac: FAISS and PyTorch together

On macOS, loading FAISS and PyTorch in the same Python process can crash with an OpenMP error, because each brings its own copy of the OpenMP library. Module 6 does exactly that (sentence-transformers runs on PyTorch, and the index is FAISS). If you see the error, set two environment variables in the terminal before you start Python, Jupyter or your editor:

export KMP_DUPLICATE_LIB_OK=TRUE
export OMP_NUM_THREADS=1

The first tells OpenMP to accept the second copy; it is a workaround, not a fix. The second keeps OpenMP to one thread, which is slower on large indexes but fine for the 24 help-centre articles. You do not need it in Colab.

From notebook cells to a project

In a job you would not keep code in notebook cells. The course grows one repository, tidewell-inbox/, with one package, inbox/. Module 1 starts it: the shared data becomes inbox/data.py, cleaning goes in inbox/clean.py, and the rules in inbox/rules.py, with a tests/ folder of pytest tests. Every later module adds its piece, and the capstone joins them in inbox/pipeline.py. The project page (03-the-tidewell-inbox.md) lists which module adds which file.

10. When something goes wrong