Part 6 · Course guide

Run Locally: the Repository, Ollama and the Parts Colab Cannot Run

5 min read·25 Oct 2026

Everything in the lessons runs in Colab. This page is for three other things: running the lessons on your own computer, using Ollama, and running the parts of the design that need real servers (PostgreSQL with pgvector, Qdrant, Redis, an RQ worker, the FastAPI service, Langfuse, Temporal). It is the course's one "run locally" page.

Status. The facts about Ollama, Langfuse, Temporal, TGI, vLLM and SGLang on this page were read on their official pages on 18 October 2026; the install links and shell commands are each tool's standard ones. The Python pins were resolved together in a dry run on that day. The Docker Compose stack, Temporal, Langfuse and the GPU serving commands were not run in the course; the labs that use them say so.

1. Install

Python 3.12 (3.13 also works) from python.org. Older than 3.12 will not work: the pinned numpy needs 3.12 and langfuse needs 3.11. Check with python --version (on some systems python3 --version).
A code editor. VS Code with the Python and Jupyter extensions is a good choice; others work the same way.
Docker Desktop (or Docker Engine with the Compose plugin) from docs.docker.com/get-docker, for Modules 9 (the Docker sandbox), 10 (the stack) and 13 (Langfuse).
Ollama, if you are using it, from ollama.com/download. Download the course's model once, in a terminal:

bash

ollama pull llama3.2:3b

That is a download of about 2.0 GB (Ollama's library page, 18 October 2026). A small local model makes more mistakes than the hosted ones; the golden-set lessons measure how many.

2. Create the project folder

bash

mkdir northdesk-platform
cd northdesk-platform
python -m venv .venv

Switch the virtual environment on:

bash

# macOS and Linux
source .venv/bin/activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

Create requirements.txt with the course's pins (the Module 1 lab writes the same file):

text

pytest==9.1.1
tiktoken==0.14.0
pydantic==2.14.1
numpy==2.5.4
pandas==3.0.6
langchain==1.4.6
langchain-core==1.6.12
langgraph==1.2.16
langgraph-checkpoint-sqlite==3.1.2
langchain-groq==1.1.3
langchain-google-genai==4.4.2
langchain-ollama==1.1.0
sentence-transformers==6.1.0
huggingface-hub==1.33.0
transformers==5.20.1
faiss-cpu==1.15.1
chromadb==1.5.9
qdrant-client==1.19.1
rank-bm25==0.2.2
litellm==1.106.0
redis==8.1.0
fakeredis==2.40.0
rq==2.12.0
mcp==2.3.0
a2a-sdk==1.2.2
fastapi==0.143.5
httpx==0.28.1
uvicorn==0.54.0
psycopg[binary]==3.3.6
pgvector==0.5.1
opentelemetry-api==1.45.1
opentelemetry-sdk==1.45.1
opentelemetry-exporter-otlp==1.45.1
arize-phoenix-otel==0.17.2
openinference-instrumentation-langchain==0.1.79
langfuse==5.0.2
temporalio==1.34.0
docker==7.2.0
python-dotenv==1.2.4

Then install:

bash

pip install -r requirements.txt

This list resolved with no conflict in a dry run on 18 October 2026 (Python 3.12, Linux, 193 packages). It has not been installed on every operating system; if pip reports a conflict on yours, keep the pins of the libraries the current module uses and let pip choose the rest. arize-phoenix (the trace viewer itself, a large install) and ragas are left out; Module 13 installs them when it needs them.

3. Keys on your own computer

Ollama needs no key. For Groq or Gemini, create a file named .env in the project folder:

text

GROQ_API_KEY=paste-your-key-here
GOOGLE_API_KEY=paste-your-key-here

and add these two lines at the very top of the setup cell:

python

from dotenv import load_dotenv
load_dotenv()

Code walkthrough

load_dotenv() reads .env and puts each line into the environment, which is where LangChain looks for keys. The setup cell's if block then finds the key already set and skips the Colab-only part.

Never share .env or commit it to Git; the Module 1 lab puts it in .gitignore.

4. Run the lessons

Open the folder in your editor and create a notebook, for example module-03-lesson-1.ipynb. Select the .venv Python when asked. Paste each code block into a cell and run it, as in Colab. Skip the %pip install cells, because you installed the libraries already, and set PROVIDER = "ollama" in the setup cell if you are using Ollama. Lines starting with ! (such as !pytest -q) work in a notebook; in a terminal, type the command without the !.

On a Mac, FAISS and PyTorch together can crash with an OpenMP error, because each brings its own copy of the OpenMP library (recorded in this series' earlier courses). If you see it, set these in the terminal before starting Python or your editor:

bash

export KMP_DUPLICATE_LIB_OK=TRUE
export OMP_NUM_THREADS=1

The first tells OpenMP to accept the second copy; it is a workaround, not a fix. The second keeps it to one thread, which is fine for Northdesk's 471 documents.

5. The service stack (Module 10 lab; not run in the course)

The Module 10 lab writes a docker-compose.yml that starts the parts Colab replaces with in-process stand-ins:

Service	Image (from the project's docs)	Replaces in Colab
`api`	built from the repository's `Dockerfile` (FastAPI under uvicorn)	`TestClient`
`worker`	the same image, running an RQ worker	`SimpleWorker` on fakeredis
`redis`	the official Redis image	fakeredis
`postgres`	a PostgreSQL image with the pgvector extension	SQLite and FAISS
`qdrant`	the official Qdrant image	`QdrantClient(":memory:")`

Start and stop it with:

bash

docker compose up -d
docker compose ps
docker compose down

The lab gives the exact file, with image tags checked on the day it is written. Until then this table is the plan, not a tested stack.

6. Langfuse self-hosted (Module 13; not run in the course)

Langfuse's self-hosting page (read 18 October 2026) describes a web container and a worker container backed by PostgreSQL, ClickHouse, Redis or Valkey, and S3 or blob storage, and offers a Docker Compose setup for a laptop. Module 13 lesson 2 gives the commands from that page on the day it is written. The open-source version is MIT licensed except its ee folders; some features need a licence key.

7. Temporal (Module 8; not run in the course)

temporalio needs a Temporal server. Module 8 lesson 4 gives the local development server command from Temporal's documentation on the day it is written, and tries the SDK's time-skipping test environment (WorkflowEnvironment.start_time_skipping(), which downloads a test server and, per the SDK's README, does not work on ARM for Linux or Windows).

8. GPU serving (Module 7; not run in the course)

vLLM 0.32.0 and SGLang 0.5.22 serve open-weight models on GPUs behind an OpenAI-compatible API. Module 7 lesson 3 gives the serve command and a client call from each project's documentation, and states that they were not run (the course has no GPU). Text Generation Inference (TGI) is archived: its repository has been read-only since 21 March 2026, and its notice points to vLLM, SGLang, llama.cpp and MLX.

9. From notebook cells to a project

In a job you would not keep code in notebook cells. The course grows one repository, northdesk-platform/, with one package, northdesk/, a service/ folder, an evals/ folder, a docs/ folder for the design documents and a tests/ folder. The project page (03-northdesk.md) lists which module adds which file.