CourseAI / ML Engineer: The Complete Course · Module 11: Monitoring and Reliability · part 68 of 93
Part 68 · Module 11: Monitoring and Reliability

Lesson 2: Running a live model: alerts, dashboards, incidents

29 min read·
Module 11 · Lesson 2 · About 60 minutes · Needs: lesson 1 (the monitors and the replay), Module 9 (the service), Module 8 lesson 3 (latency percentiles), Module 10 lesson 2 (shadow deployment). The model is trained in one cell here.

The situation

Week 11, Thursday. Lesson 1 found the grams on 7 September. Rhys owns the pager and sets the terms: "Page me once for 7 September. Never page me because it rained in Inverness." Imran wants the training and GPU bills tracked. Adaeze wants a postmortem template before anyone writes one.

The running example is still Monday 7 September 2026 (simulated): the day the alerts must catch, the incident to respond to, and the postmortem's subject.

WHERE THIS FITS The Kestrel system this lesson SHARED SERVICES Depot scans parcels, routes Ingestion nightly batch Validation Pandera checks Data lake Parquet, DuckDB Experiments MLflow runs Training LightGBM, Optuna Online store Feast, Redis Features point in time Gates vs the champion Registry staged versions Serving API FastAPI, ONNX Consumers vans, customers Retraining trigger, budget Alerting on call Monitoring drift, Evidently Late labels a day later Orchestrator Prefect flows Versions DVC, Git CI/CD GitHub Actions Containers Docker images
This lesson adds alerting, the model-health dashboard and incident handling (blue) around lesson 1's monitoring and the scoring service; the rest is grey for now.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan