CourseAI / ML Engineer: The Complete Course · Module 11: Monitoring and Reliability · part 69 of 93
Part 69 · Module 11: Monitoring and Reliability

Lab: September replayed through a monitoring job

170 min read·
Module 11 · Lab · About 60 minutes · Needs: lessons 1 and 2 of this module, and the Module 10 lab (or nothing: Step 1 rebuilds Modules 1 to 10)

What you are building

The monitoring job Numbers per day, decisions per morning Active flow Background or async 1 Processing step number Optional or fallback Prediction log model v2 ONNX, as served labels joined by parcel monitor_flow.py Prefect day_row no labels needed labels_row matured labels tune limits from quiet days plan, 05:00 flow page, ticket, quiet People page Rhys, on call ticket office hours dashboard display only Documents dashboard.md postmortem ADR-008 1 2 3
The prediction log of model v2 and the matured labels (blue, gold) feed the monitoring job (teal), which turns each day's numbers into a page, a ticket or a dashboard line with limits tuned on quiet days; the lab then writes the three documents; circles give the order.

Nothing crashed in September, and nobody knew the model had gone wrong. This lab builds the job that would have known: pipelines/monitor_flow.py reads every scored parcel, computes the lessons' health numbers for each ship day and each morning's matured labels, and turns them into pages, tickets and dashboard lines with thresholds tuned on history. Then it writes the three documents that make monitoring a practice: the dashboard specification, the postmortem for 7 September, and ADR-008.

The lessons monitored a stand-in model trained in the notebook. The lab monitors the real one: model v2 from artifacts/model.onnx, served the way the service ran in September, before contract 1.1's grams fix reached it. Every number below is v2's, replayed from 6 July to 27 September. No cell calls a hosted model.

Open a new Colab notebook, call it module-11-lab.ipynb, and run each cell in order.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan