Monitoring Predictions: Logging Inputs and Outputs
Learners will add prediction logging to the API, discuss data drift and model staleness, and sketch the retraining trigger and deployment pipeline.
Why Monitor a Deployed Model?
A model that performed excellently at deployment can silently degrade as the real world changes. Customer behaviour shifts, new product categories appear, sensors drift, economic conditions change — all causing the input distribution to diverge from training data. Without monitoring, you discover model failure when users complain or business metrics drop. Prediction monitoring catches degradation early, before it impacts users.
What to Log: The Three W's
Every prediction event should log: When (timestamp), What (input features and the model's output/prediction/probability), and ideally Whether (the eventual ground truth label, once available). The input features are needed for data drift detection; the outputs enable confidence monitoring; the ground truth enables accuracy monitoring over time.
All lessons in this course
- Saving Models with joblib and pickle
- Versioning Models: Why Filenames and Metadata Matter
- Serving Predictions with a FastAPI Endpoint
- Monitoring Predictions: Logging Inputs and Outputs