Lab: Model v2 in ONNX, benchmarked

Module 8 · Lab · About 50 minutes · Needs: lessons 1 to 3 of this module, and the Module 7 lab (or nothing: Step 1 rebuilds Modules 1 to 7)

What you are building

Rhys's question is the second of the course's five: "What is p99, and where does the time go?" The lessons answered it for stand-ins: a LightGBM on 21 numbers and a small network, both trained in the lesson. This lab answers it for model v2 as the Module 6 lab built it: Module 3's encoder, LightGBM and Platt scaling, reading the 18 columns known at collection.

The lab exports the whole of v2, encoder and calibration included, to one ONNX graph, artifacts/model.onnx, and checks it parcel for parcel against v2 in Python. Then it benchmarks both with Lesson 3's harness, finds where a single parcel's time goes, prices a thousand predictions, and writes reports/benchmark.md and docs/adr/ADR-005-inference-runtime.md. The code lives in kestrel/optimise/onnx_model.py. No cell calls a hosted model.

Open a new Colab notebook, call it module-08-lab.ipynb, and run each cell in order. A CPU runtime is enough.