Lesson 3: Containers, GPUs and scaling a slow resource
24 min read·
Module 10 · Lesson 3 · About 50 minutes · Needs: Module 3 lessons 1 and 3 (peak RPS, GPU sizing) and Module 7 lesson 3 (the self-hosting decision). No key and no GPU
The situation
Week 10 at Northdesk. Module 7 kept Ask on hosted models and promised to revisit self-hosting after a benchmark. Funmi wants the plan ready before that day: what goes into each container image, whether Northdesk needs Kubernetes, how GPU replicas scale when each one is slow to start and costs money every hour, and how the platform knows a model server is ready. Her constraint, from Module 2: the team has no GPU operations experience.
The running example is Monday 28 September 2026 at Joel's target scale. At 07:00 UTC UK stores open and Ask jumps from 13.9 to 30.6 requests a second (RPS). One self-hosted gpt-oss-20b replica on one GPU takes 17.5 RPS. Does the fleet keep up?