CourseAI / ML Engineer: The Complete Course · Module 7: Training at Scale · part 44 of 93
Part 44 · Module 7: Training at Scale

Lesson 1: What a GPU holds, how fast it feeds, what it costs

21 min read·
Module 7 · Lesson 1 · About 50 minutes · Needs: what a parameter, a batch and an optimiser are (Module 4 lesson 1), and a PyTorch training loop (Module 4 lesson 2). No GPU needed; a free Colab T4 adds one optional measurement.

The situation

Week 7 at Kestrel. Imran sends one line: "What does one training run cost, and do we need to buy GPU time?" Rhys owns the GPU question and wants the answer as arithmetic, not opinion.

Kestrel has two training jobs. The late-risk network from Module 4 has 3,521 parameters and learns from 86,014 training parcels. The message model that Module 12 adapts is SmolLM2-135M, an open model with 134,515,008 parameters. This lesson budgets both, in bytes, FLOPs, hours and dollars, and traces them side by side.

WHERE THIS FITS The Kestrel system this lesson SHARED SERVICES Depot scans parcels, routes Ingestion nightly batch Validation Pandera checks Data lake Parquet, DuckDB Experiments MLflow runs Training LightGBM, Optuna Online store Feast, Redis Features point in time Gates vs the champion Registry staged versions Serving API FastAPI, ONNX Consumers vans, customers Retraining trigger, budget Alerting on call Monitoring drift, Evidently Late labels a day later Orchestrator Prefect flows Versions DVC, Git CI/CD GitHub Actions Containers Docker images
This lesson sizes the hardware under training and experiments (blue); the rest of Kestrel's system is grey for now.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan