Machine Learning Academy · 课时

打包、记录与展示最终模型

学习者将序列化获胜流程，编写模型卡片，记录训练数据、性能、局限性和公平性考量，并完成五分钟演示。

打包、记录与展示最终模型 是 CoddyKit 上的免费 Machine Learning Academy 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Packaging and Documentation Matter

A model that lives only in a Jupyter notebook is not a deliverable — it is a prototype. Packaging means serialising the trained pipeline into a portable artifact that can be loaded and used without the original training code. Documentation means producing a model card that records what the model does, what data it was trained on, how well it performs, where it might fail, and who should use it. Together, packaging and documentation are what transform research into responsible production software.

Serialising the Final Pipeline with joblib

joblib.dump serialises a fitted scikit-learn Pipeline (including all preprocessors and the model) to a single file. This file can be loaded in any Python environment with the same scikit-learn version installed. Use a structured filename that encodes the project, model type, training date, and performance metric so you can identify any artifact without reading its metadata sidecar.

import joblib
import json
from datetime import date

# Save the fitted pipeline
model_filename = f'churn_xgboost_{date.today().isoformat()}_auc0883.pkl'
joblib.dump(best_pipeline, model_filename)
print(f'Saved pipeline: {model_filename}')

# Verify round-trip
loaded_pipeline = joblib.load(model_filename)
y_reloaded = loaded_pipeline.predict_proba(X_test[:5])[:, 1]
y_original = best_pipeline.predict_proba(X_test[:5])[:, 1]
print('Predictions match after reload:', all(y_reloaded == y_original))

Writing a JSON Metadata Sidecar

Alongside the serialised model, write a .json metadata file that documents the artifact. This sidecar enables governance tooling to index models without loading the binary. Include: training date, dataset version or hash, scikit-learn version, CV performance metrics, feature names, and the SHA-256 hash of the model file to detect tampering or corruption.

import hashlib, json, sklearn, joblib, os
from datetime import date

def sha256_file(path):
    h = hashlib.sha256()
    with open(path, 'rb') as f:
        for chunk in iter(lambda: f.read(8192), b''):
            h.update(chunk)
    return h.hexdigest()

metadata = {
    'model_type': 'XGBClassifier',
    'task': 'binary_classification',
    'target': 'churn_90d',
    'training_date': date.today().isoformat(),
    'sklearn_version': sklearn.__version__,
    'feature_names': list(feature_names),
    'cv_auc_mean': 0.883,
    'cv_auc_std': 0.007,
    'test_auc': 0.879,
    'sha256': sha256_file(model_filename)
}
with open(model_filename.replace('.pkl', '_metadata.json'), 'w') as f:
    json.dump(metadata, f, indent=2)
print(json.dumps(metadata, indent=2))

Writing a Model Card

A model card (Mitchell et al., 2019) is a short document — typically 1-2 pages — that describes a model for the people who will use or be affected by it. It is structured into sections: model details, intended use, metrics, training data, evaluation data, ethical considerations, and caveats/recommendations. Model cards are now required by the EU AI Act for high-risk AI systems and recommended by major ML providers.

model_card_template = '''
# Model Card: Customer Churn Predictor v1.0

## Model Details
- Type: XGBoost binary classifier inside a scikit-learn Pipeline
- Task: Predict 90-day customer churn
- Version: 1.0.0 | Training date: 2026-06-25

## Intended Use
- Primary use: Nightly batch scoring to generate a churn risk score per customer
- Out-of-scope: Real-time scoring, non-B2C segments, churn windows != 90 days

## Metrics
- 5-fold CV AUC: 0.883 +/- 0.007
- Held-out test AUC: 0.879
- Precision@threshold=0.4: 0.67 | Recall@threshold=0.4: 0.82

## Training Data
- Source: orders DB + CRM, 2024-01-01 to 2026-04-30
- Samples: 120,000 customers | Churn rate: 8.3%

## Ethical Considerations
- Age and region features audited for demographic parity (difference < 0.03)
- No direct use of protected attributes

## Caveats
- Performance may degrade if product catalogue changes significantly
- Retraining recommended if test AUC drops below 0.85 in monitoring
'''
print(model_card_template)

Generating Performance Artefacts

Attach key visualisations to the model card as evidence of performance. Standard artefacts include: the ROC curve with AUC annotated, the precision-recall curve (more informative for imbalanced classes), the confusion matrix at the deployment threshold, and a SHAP beeswarm plot for global feature importance. Save each as a PNG to include in the model card repository alongside the binary artifact.

from sklearn.metrics import RocCurveDisplay, PrecisionRecallDisplay, ConfusionMatrixDisplay
import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 3, figsize=(15, 4))

# ROC curve
RocCurveDisplay.from_predictions(y_test, y_proba, ax=axes[0], name='XGBoost')
axes[0].set_title('ROC Curve')

# Precision-Recall curve
PrecisionRecallDisplay.from_predictions(y_test, y_proba, ax=axes[1], name='XGBoost')
axes[1].set_title('Precision-Recall Curve')

# Confusion matrix at threshold 0.4
y_pred_thresh = (y_proba >= 0.4).astype(int)
ConfusionMatrixDisplay.from_predictions(y_test, y_pred_thresh, ax=axes[2], display_labels=['retained', 'churned'])
axes[2].set_title('Confusion Matrix @ threshold=0.4')

plt.tight_layout()
plt.savefig('model_card_performance.png', dpi=150)

Versioning with Git and DVC

Model artifacts and datasets should be version-controlled alongside code. Git tracks code and metadata JSON files. DVC (Data Version Control) tracks large binary files (model pickles, datasets) separately in object storage (S3, GCS) while storing only a lightweight pointer in Git. This gives you a full audit trail: for any git commit, you can reproduce the exact model artifact and dataset that produced it.

# DVC workflow (shell commands — not Python)
# pip install dvc[s3]

# Initialise DVC in the repo
# dvc init

# Add model artifact to DVC tracking
# dvc add churn_xgboost_2026-06-25_auc0883.pkl
# git add churn_xgboost_2026-06-25_auc0883.pkl.dvc .gitignore
# git commit -m 'Add XGBoost churn model v1.0'

# Push artifact to S3
# dvc remote add myremote s3://my-ml-artifacts/churn-model
# dvc push

# To reproduce: checkout a git commit, then:
# dvc pull  # downloads the exact artifact for that commit
print('DVC enables git-compatible versioning of large binary model artifacts.')

Packaging as a Python Module

For reuse across multiple services, wrap the model in a lightweight Python package with a clean prediction interface. Define a predict(features: dict) -> dict function that loads the model once at module import, validates input, runs prediction, and returns a structured response. This decouples the consumer from the serialisation format and makes the interface testable independently of the model binary.

# churn_model/predictor.py
import joblib
import numpy as np
from pathlib import Path

_MODEL_PATH = Path(__file__).parent / 'artifacts' / 'churn_xgboost_latest.pkl'
_PIPELINE = None

def _load():
    global _PIPELINE
    if _PIPELINE is None:
        _PIPELINE = joblib.load(_MODEL_PATH)
    return _PIPELINE

def predict(features: dict) -> dict:
    '''Return churn probability for a single customer feature dict.'''
    pipeline = _load()
    # Convert dict to 2D array in correct feature order
    feature_order = pipeline.feature_names_in_
    X = np.array([[features[col] for col in feature_order]])
    proba = pipeline.predict_proba(X)[0, 1]
    return {'churn_probability': float(proba), 'churn_flag': proba >= 0.4}

Writing the Five-Minute Demo

A capstone presentation must communicate the model's value to non-technical stakeholders in five minutes. Structure it as: (1) Problem — cost of churn; (2) Solution — what the model predicts; (3) Results — AUC and the business impact estimate; (4) Explanation — SHAP top features to build trust; (5) Next steps — A/B test plan and retraining schedule. Lead with business value, not algorithm details.

demo_outline = [
    ('Slide 1 — Problem', '30s',
     'Company loses $2.4M/year to churn. We predict who will churn 90 days in advance.'),
    ('Slide 2 — Data', '30s',
     '120k customers, 3 data sources, 8.3% churn rate.'),
    ('Slide 3 — Model',  '45s',
     'XGBoost tournament winner. AUC=0.88: catches 82% of churners with 67% precision.'),
    ('Slide 4 — Explanation', '60s',
     'Top 3 signals: days since last purchase, support tickets, plan type. SHAP beeswarm.'),
    ('Slide 5 — Business impact', '45s',
     'At 15% retention rate from targeted offers: estimated $360k annual recovery.'),
    ('Slide 6 — Next steps', '30s',
     'A/B test for 90 days. Weekly retraining. Fairness audit before full rollout.')
]
for slide, timing, content in demo_outline:
    print(f'{slide} [{timing}]: {content}')

Testing the Packaged Model

Before declaring the model production-ready, write automated tests for the prediction interface. Test: predictions are in [0, 1] range, missing features raise a clear error, the model output is deterministic across calls, and performance on a small labelled fixture dataset matches the documented metrics. These tests run in CI on every code change and catch silent regressions before they reach users.

import pytest
import numpy as np
from churn_model.predictor import predict

def test_output_is_probability():
    sample = {'days_since_last_purchase': 45, 'total_spend_90d': 120.5, 'support_tickets': 2}
    result = predict(sample)
    assert 0.0 <= result['churn_probability'] <= 1.0

def test_missing_feature_raises():
    with pytest.raises(KeyError):
        predict({'days_since_last_purchase': 45})  # missing required features

def test_deterministic():
    sample = {'days_since_last_purchase': 10, 'total_spend_90d': 500.0, 'support_tickets': 0}
    r1 = predict(sample)['churn_probability']
    r2 = predict(sample)['churn_probability']
    assert r1 == r2  # model must not use randomness at inference

print('Run: pytest test_predictor.py -v')

Setting Up Monitoring Before Launch

Before flipping the production switch, configure monitoring so you know immediately if the model degrades. Log every prediction (input features, probability, threshold decision, timestamp) to a structured store. Set up dashboards tracking: prediction volume per day, mean probability over time, and false positive rate on a labelled sample. Define alert thresholds: if mean probability drops by more than 0.1 or weekly AUC drops below 0.82, trigger a retraining run automatically.

# Prediction logging middleware
import json
import time
from pathlib import Path

LOG_FILE = Path('/var/log/churn_model/predictions.jsonl')
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)

def predict_and_log(features: dict) -> dict:
    result = predict(features)
    log_entry = {
        'timestamp': time.time(),
        'customer_id': features.get('customer_id'),
        'churn_probability': result['churn_probability'],
        'churn_flag': result['churn_flag']
    }
    with open(LOG_FILE, 'a') as f:
        f.write(json.dumps(log_entry) + '\n')
    return result

print('Prediction logging writes to JSONL; ingest into Grafana or BigQuery for dashboarding.')

The Complete Deliverable Checklist

A production-ready ML deliverable consists of: model artifact (serialised pipeline + metadata JSON), model card (problem, data, metrics, fairness, caveats), prediction module (clean Python interface with input validation), automated tests (unit + integration), performance artefacts (ROC, PR curve, SHAP plot), monitoring setup (prediction logging + alert thresholds), and stakeholder presentation (5-minute demo). Together these make the model auditable, reproducible, and maintainable over its lifetime.

final_checklist = [
    '[x] Trained pipeline serialised with joblib + SHA-256 verified',
    '[x] JSON metadata sidecar with training date, metrics, and feature names',
    '[x] Model card written and reviewed by domain expert',
    '[x] SHAP global importance plot attached to model card',
    '[x] ROC + PR curve + confusion matrix artefacts generated',
    '[x] Fairness audit: demographic parity difference < 0.05',
    '[x] Prediction module with clean API and input validation',
    '[x] 100% test coverage of prediction module (pytest)',
    '[x] Prediction logging middleware deployed',
    '[x] Alert thresholds configured in monitoring dashboard',
    '[x] Stakeholder demo delivered and recorded'
]
for item in final_checklist:
    print(item)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: serialise the winning pipeline with joblib and write a JSON metadata sidecar for governance, a model card documents intended use, metrics, training data, fairness, and caveats for every stakeholder, and wrap the model in a tested Python module with logging and monitoring before deployment. You have now completed the full Machine Learning with Python track — from raw data all the way to a documented, monitored, production-ready model. Congratulations!

常见问题解答

「打包、记录与展示最终模型」课时是免费的吗？

是的 — 「打包、记录与展示最终模型」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。

「打包、记录与展示最终模型」这节课中我会学到什么？

学习者将序列化获胜流程，编写模型卡片，记录训练数据、性能、局限性和公平性考量，并完成五分钟演示。你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Machine Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「打包、记录与展示最终模型」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Machine Learning Academy 课中编写并运行代码吗？

能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Machine Learning Academy