CourseAI / ML Engineer: The Complete Course · Module 8: Model Optimisation · part 51 of 93
Part 51 · Module 8: Model Optimisation

Lesson 2: Making models faster: ONNX, batching, caching, compiling

28 min read·
Module 8 · Lesson 2 · About 55 minutes · Needs: lesson 1 (the float network and the LightGBM), attention (Module 4 lesson 3), and the Kestrel data (course-guide/06).

The situation

Week 8 at Kestrel, Tuesday 22 December 2026. Next week's scoring service runs models in ONNX Runtime. Rhys describes two loads: at 06:00 every parcel due that day arrives at once; during the day parcels arrive one at a time. "Make each model fast, and prove it gives the same answers."

The running example is again KP0104204 (simulated), the late Dartford parcel: the network gives it 0.546, the LightGBM 0.628, and every faster version must agree.

WHERE THIS FITS The Kestrel system this lesson SHARED SERVICES Depot scans parcels, routes Ingestion nightly batch Validation Pandera checks Data lake Parquet, DuckDB Experiments MLflow runs Training LightGBM, Optuna Online store Feast, Redis Features point in time Gates vs the champion Registry staged versions Serving API FastAPI, ONNX Consumers vans, customers Retraining trigger, budget Alerting on call Monitoring drift, Evidently Late labels a day later Orchestrator Prefect flows Versions DVC, Git CI/CD GitHub Actions Containers Docker images
This lesson prepares the model for the scoring service (blue): the format it is stored in and the runtime that serves it.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan