Machine Learning Academy · Lekcja

Automatyczne pipeline'y ponownego trenowania z GitHub Actions

Uczą się Państwo pisać workflow GitHub Actions uruchamiany zgodnie z harmonogramem danych, który trenuje model, ocenia go i promuje tylko wtedy, gdy przewyższa produkcyjny model bazowy.

Automatyczne pipeline'y ponownego trenowania z GitHub Actions to bezpłatna lekcja Machine Learning Academy na CoddyKit. To lekcja 4 z 4. Możesz przeczytać całą lekcję poniżej za darmo — a potem ćwiczyć ją interaktywnie w przeglądarce z wbudowanym edytorem kodu i tutorem AI dostępnym 24/7. To część ścieżki edukacyjnej Machine Learning Academy, a Twój postęp synchronizuje się między webem a aplikacją CoddyKit. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Części tej lekcji nie zostały jeszcze przetłumaczone i są wyświetlane po angielsku.

Why Automate Model Retraining?

Models trained on historical data degrade over time as real-world patterns shift. Manually retraining requires a data scientist to remember to do it, run scripts, evaluate results, and update deployment — a process prone to delays and human error. Automated retraining pipelines run on a schedule or when triggered by data arrival, automatically train a new model, evaluate it against the current champion, and promote it only if performance improves — with zero manual intervention.

GitHub Actions: Workflows and Triggers

GitHub Actions is a CI/CD platform built into GitHub. Workflows are defined in YAML files under .github/workflows/ and run in response to triggers: code pushes, pull requests, manual dispatches, or cron schedules. Each workflow consists of jobs (groups of steps) that run on hosted or self-hosted runners. For ML pipelines, a scheduled cron trigger on a cloud GPU runner is the most common pattern.

# .github/workflows/retrain.yml (YAML structure shown)
# name: Model Retraining Pipeline
#
# on:
#   schedule:
#     - cron: '0 2 * * 1'  # every Monday at 2 AM UTC
#   workflow_dispatch:       # also allow manual trigger
#
# jobs:
#   retrain:
#     runs-on: ubuntu-latest
#     steps:
#       - uses: actions/checkout@v4
#       - name: Set up Python
#         uses: actions/setup-python@v5
#         with:
#           python-version: '3.11'
print('Workflow YAML structure shown above.')

Cron Syntax for Scheduling

GitHub Actions cron expressions follow standard Unix cron format: minute hour day month weekday. Common ML retraining schedules include weekly retraining (when new weekly data batches are available) and nightly retraining for fast-moving domains. Use workflow_dispatch alongside cron so data scientists can also trigger retraining manually when they push a new training script or detect drift.

# Common cron schedule examples for ML pipelines
# '0 2 * * 1'      -- Every Monday at 02:00 UTC (weekly)
# '0 3 * * *'      -- Every day at 03:00 UTC (nightly)
# '0 */6 * * *'    -- Every 6 hours
# '0 2 1 * *'      -- First day of every month at 02:00

# All times are UTC in GitHub Actions
# Use https://crontab.guru to verify cron expressions

# Example in YAML:
# on:
#   schedule:
#     - cron: '0 3 * * *'   # nightly at 3 AM UTC
#   workflow_dispatch:

print('Cron schedule examples shown above.')

Checkout, Setup Python, Install Deps

The first steps in every ML workflow checkout the repository code, set up the required Python version, and install dependencies from requirements.txt. Use dependency caching with actions/cache to avoid reinstalling packages on every run — this can reduce pipeline duration from 10 minutes to under 2 minutes by reusing the cached pip packages when requirements.txt has not changed.

# .github/workflows/retrain.yml -- steps section
# steps:
#   - uses: actions/checkout@v4
#
#   - name: Set up Python 3.11
#     uses: actions/setup-python@v5
#     with:
#       python-version: '3.11'
#
#   - name: Cache pip packages
#     uses: actions/cache@v4
#     with:
#       path: ~/.cache/pip
#       key: pip-${{ hashFiles('requirements.txt') }}
#
#   - name: Install dependencies
#     run: pip install -r requirements.txt

print('Workflow steps shown above.')

Secrets Management in GitHub Actions

API keys, database passwords, and cloud credentials must never be hard-coded in workflow files. Store them as GitHub Secrets (Settings → Secrets → Actions) and reference them as ${{ secrets.SECRET_NAME }}. They are masked in logs and unavailable to forked pull requests. For ML pipelines, store MLflow credentials, cloud storage keys, and database connection strings as secrets.

# .github/workflows/retrain.yml -- env section with secrets
# jobs:
#   retrain:
#     runs-on: ubuntu-latest
#     env:
#       MLFLOW_TRACKING_URI: ${{ secrets.MLFLOW_TRACKING_URI }}
#       AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
#       AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
#       DATABASE_URL: ${{ secrets.PROD_DATABASE_URL }}
#
# In train.py, read normally:
import os
DB_URL = os.getenv('DATABASE_URL')
MLFLOW_URI = os.getenv('MLFLOW_TRACKING_URI')
print('Config loaded from environment.')

The Training Step

The core step runs your Python training script. The script should accept configuration via environment variables (set in the workflow), log all parameters and metrics to MLflow, and write the new model to the registry as a new version in None stage. Keep the script idempotent: running it twice with the same data should produce a new registry version each time without corrupting existing versions.

# train.py -- called by GitHub Actions
import os
import mlflow
import mlflow.sklearn
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import f1_score

mlflow.set_tracking_uri(os.getenv('MLFLOW_TRACKING_URI', 'http://localhost:5000'))
mlflow.set_experiment('automated_retraining')

with mlflow.start_run(run_name='scheduled_retrain'):
    n_est = int(os.getenv('N_ESTIMATORS', '200'))
    lr = float(os.getenv('LEARNING_RATE', '0.05'))
    mlflow.log_param('n_estimators', n_est)
    mlflow.log_param('learning_rate', lr)

    clf = GradientBoostingClassifier(n_estimators=n_est, learning_rate=lr, random_state=42)
    # clf.fit(X_train, y_train)
    # f1 = f1_score(y_test, clf.predict(X_test))
    # mlflow.log_metric('test_f1', f1)
    # mlflow.sklearn.log_model(clf, 'model', registered_model_name='SentimentClassifier')
    print('Training step complete.')

The Evaluation Step: Beat the Champion

After training, an evaluation step compares the new model against the current Production champion. If the new model wins, it proceeds to promotion; otherwise the pipeline marks the new version as rejected (tag it with status=rejected) and keeps the champion in Production. Use a GitHub Actions output variable to pass the promotion decision between steps.

# evaluate.py -- compare new version against production
from mlflow.tracking import MlflowClient
import sys

client = MlflowClient()
MODEL_NAME = 'SentimentClassifier'

def get_f1(run_id):
    return client.get_run(run_id).data.metrics.get('test_f1', 0)

# Get latest registered version (just added by train.py)
new_versions = client.get_latest_versions(MODEL_NAME, stages=['None'])
new_v = sorted(new_versions, key=lambda v: int(v.version))[-1]
new_f1 = get_f1(new_v.run_id)

prod_versions = client.get_latest_versions(MODEL_NAME, stages=['Production'])
if prod_versions:
    prod_f1 = get_f1(prod_versions[0].run_id)
    print(f'Champion F1: {prod_f1:.4f}  Challenger F1: {new_f1:.4f}')
    should_promote = new_f1 > prod_f1
else:
    print('No champion found.')
    should_promote = True

print('should_promote=' + str(should_promote).lower())
sys.exit(0 if should_promote else 1)  # exit code drives next step

The Promotion Step: Conditional Execution

Use GitHub Actions' if: success() and if: failure() conditions on steps or jobs to implement conditional promotion. After the evaluation step, a promotion step runs only if evaluation succeeded (the challenger won). Use a separate notification step that always runs to send a Slack or email alert with the pipeline outcome, regardless of whether promotion happened.

# promote.py -- transition new version to Production
from mlflow.tracking import MlflowClient

client = MlflowClient()
MODEL_NAME = 'SentimentClassifier'

new_versions = client.get_latest_versions(MODEL_NAME, stages=['None'])
new_v = sorted(new_versions, key=lambda v: int(v.version))[-1]

client.transition_model_version_stage(
    name=MODEL_NAME,
    version=new_v.version,
    stage='Production',
    archive_existing_versions=True
)
print(f'Version {new_v.version} promoted to Production.')

# In the workflow YAML:
# - name: Promote model
#   if: success()  # only runs if evaluate.py exits with 0
#   run: python promote.py

Data Download in the Pipeline

Retraining pipelines need fresh data. Add a data download step before training that fetches the latest data from S3, a database, or a data lake. Use versioned data snapshots (tagged with a date) rather than always pulling the latest to maintain reproducibility: if a model performs poorly, you need to reproduce the exact training data it saw. Log the data version as an MLflow parameter alongside model hyperparameters.

# download_data.py -- fetch latest training data from S3
import boto3
import os
from datetime import date

DATA_DATE = os.getenv('DATA_DATE', str(date.today()))  # e.g., 2024-06-15
BUCKET = os.getenv('S3_BUCKET', 'my-ml-data')
KEY = f'datasets/sentiment/{DATA_DATE}/train.parquet'

s3 = boto3.client('s3')
os.makedirs('/tmp/data', exist_ok=True)
s3.download_file(BUCKET, KEY, '/tmp/data/train.parquet')
print(f'Downloaded data for {DATA_DATE}')

# In train.py, log data_date as a parameter:
# mlflow.log_param('data_date', DATA_DATE)

Notifications with Slack Webhooks

An automated pipeline without notifications is a black box. Add a final notification step that always runs (using if: always()) and posts a summary to Slack or sends an email. Include the run outcome (promoted or rejected), the new version number, the F1 scores of both champion and challenger, and a direct link to the MLflow run for drill-down. Good notifications turn silent automation into a collaborative data science workflow.

import os
import requests
import json

SLACK_WEBHOOK = os.getenv('SLACK_WEBHOOK_URL')
PROMOTED = os.getenv('PROMOTED', 'false') == 'true'
NEW_VERSION = os.getenv('NEW_VERSION', 'N/A')
NEW_F1 = os.getenv('NEW_F1', 'N/A')

status_icon = ':white_check_mark:' if PROMOTED else ':x:'
status_text = 'Promoted to Production' if PROMOTED else 'Champion retained'

message = {
    'text': (f'{status_icon} *ML Retraining Pipeline*\n'
             f'Model: SentimentClassifier v{NEW_VERSION}\n'
             f'Status: {status_text}\n'
             f'New F1: {NEW_F1}')
}
if SLACK_WEBHOOK:
    requests.post(SLACK_WEBHOOK, data=json.dumps(message))
    print('Slack notification sent.')

Complete Workflow YAML Overview

A complete automated retraining workflow has five sequential jobs: download_data (fetch latest dataset), train (run training script, log to MLflow), evaluate (compare vs production champion), promote (transition stage if challenger wins), and notify (always-run Slack alert). Each job passes outputs to the next using GitHub Actions job outputs or shared artifact files. The entire pipeline runs unattended on a weekly schedule.

# Full workflow summary (pseudo-YAML)
# name: Weekly Model Retraining
# on:
#   schedule: [{cron: '0 3 * * 1'}]
#   workflow_dispatch: {}
# jobs:
#   download_data:
#     runs-on: ubuntu-latest
#     steps: [checkout, setup-python, install, run download_data.py]
#   train:
#     needs: download_data
#     steps: [checkout, setup-python, install, run train.py]
#   evaluate:
#     needs: train
#     steps: [checkout, setup-python, install, run evaluate.py]
#   promote:
#     needs: evaluate
#     if: success()
#     steps: [checkout, setup-python, install, run promote.py]
#   notify:
#     needs: [evaluate, promote]
#     if: always()
#     steps: [run notify.py]
print('Full workflow YAML structure shown above.')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: GitHub Actions cron triggers run retraining on a schedule without manual intervention, evaluation scripts implement the champion/challenger pattern so only better models are promoted, and secrets management keeps credentials out of workflow YAML files using encrypted GitHub Secrets. Next up we explore data drift detection — identifying when the input distribution shifts away from training conditions, which is the most common cause of production model degradation.

Często zadawane pytania

Czy lekcja „Automatyczne pipeline'y ponownego trenowania z GitHub Actions” jest bezpłatna?

Tak — pełny tekst „Automatyczne pipeline'y ponownego trenowania z GitHub Actions” jest dostępny za darmo tutaj w sieci. Aby ćwiczyć ją interaktywnie (wbudowany edytor kodu i tutor AI dostępny 24/7) i odblokować resztę kursu Machine Learning Academy, przejdź na CoddyKit PRO. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Co nauczysz się w „Automatyczne pipeline'y ponownego trenowania z GitHub Actions”?

Uczą się Państwo pisać workflow GitHub Actions uruchamiany zgodnie z harmonogramem danych, który trenuje model, ocenia go i promuje tylko wtedy, gdy przewyższa produkcyjny model bazowy. Ćwiczysz Machine Learning Academy z praktycznym kodem, który uruchamiasz bezpośrednio w przeglądarce, a tutor AI dostępny 24/7 odpowiada na Twoje pytania podczas pracy nad lekcją.

Czy potrzebuję doświadczenia, aby zacząć Machine Learning Academy?

Nie wymagamy żadnego doświadczenia. Machine Learning Academy w CoddyKit jest strukturyzowany dla początkujących i zaawansowanych użytkowników, więc możesz zacząć tutaj lub od początku i uczyć się w swoim tempie. To lekcja 4 z 4.

Ile czasu zajmuje lekcja „Automatyczne pipeline'y ponownego trenowania z GitHub Actions”?

Większość lekcji CoddyKit trwa około 5–10 minut. Każda lekcja to mały, interaktywny krok, dzięki czemu robisz systematyczne postępy i zawsze wracasz dokładnie do tego samego miejsca — na webie i w aplikacji.

Czy mogę pisać i uruchamiać kod w tej lekcji Machine Learning Academy?

Tak. Każda lekcja Machine Learning Academy zawiera wbudowany edytor kodu, więc piszesz i uruchamiasz prawdziwy kod bezpośrednio w przeglądarce i od razu otrzymujesz sprzężenie zwrotne od AI — bez konfiguracji na komputerze.

Wszystkie lekcje w tym kursie

Śledzenie eksperymentów za pomocą MLflow: parametry, metryki i artefakty
Powtarzalne środowiska z Dockerem na potrzeby ML
Rejestr modeli: staging, produkcja i archiwizacja
Automatyczne pipeline'y ponownego trenowania z GitHub Actions

← Powrót do Machine Learning Academy