머신러닝을 위한 Docker 재현 가능 환경
학습자는 Python, 버전이 고정된 머신러닝 라이브러리를 설치하고 학습 스크립트를 복사하는 Dockerfile을 작성한 뒤, 컨테이너를 빌드하고 실행하여 비트 단위의 재현 가능성을 확인합니다.
머신러닝을 위한 Docker 재현 가능 환경은(는) CoddyKit의 무료 Machine Learning Academy 강의입니다. 이것은 4개 중 2번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Machine Learning Academy 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Machine Learning Academy 강의에는 총 4개의 강의가 포함되어 있습니다.
이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.
The Reproducibility Problem in ML
A model trained on your laptop may produce different results on a colleague's machine because of different Python versions, library versions, or system libraries. Docker solves this by packaging your entire environment — Python, ML libraries, and your training script — into a container that runs identically on any machine. Containerised ML training is the foundation of reproducible research and reliable CI/CD pipelines.
Docker Concepts: Image, Container, Layer
A Docker image is a read-only template defined by a Dockerfile. It consists of stacked layers: each instruction in the Dockerfile adds a layer. A container is a running instance of an image. Images are portable and versioned via tags like my-ml-image:v1.2. The key insight is that the same image tag produces bitwise-identical runtime environments everywhere, from your laptop to cloud GPU instances.
# Basic Docker commands for ML workflows
# Build an image from Dockerfile in current directory
# docker build -t my-ml-trainer:v1 .
# Run a training script inside the container
# docker run --rm --gpus all my-ml-trainer:v1 python train.py
# List running containers
# docker ps
# Inspect image layers (shows what changed at each step)
# docker history my-ml-trainer:v1Writing a Dockerfile for ML Training
A Dockerfile is a text file with sequential instructions that Docker executes to build an image. Start from an official Python base image, set the working directory, copy a requirements.txt, install dependencies, then copy your training code. The FROM instruction specifies the base image; for GPU training use NVIDIA's CUDA-enabled base images.
# Dockerfile
# FROM python:3.11-slim
#
# WORKDIR /app
#
# # Copy and install dependencies first (layer caching)
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt
#
# # Copy training code
# COPY train.py .
# COPY data/ data/
#
# # Default command
# CMD ['python', 'train.py']
print('Dockerfile structure shown above (Python comment).')Pinning Dependencies in requirements.txt
Reproducibility requires pinned versions in requirements.txt. Use exact version specifiers (==) rather than minimum versions (>=). Generate a complete pinned requirements file with pip freeze > requirements.txt after testing your environment. For ML projects, always pin scikit-learn, numpy, pandas, and any framework versions, as minor version changes can alter model behaviour.
# requirements.txt (pinned versions for reproducibility)
# scikit-learn==1.4.2
# numpy==1.26.4
# pandas==2.2.1
# matplotlib==3.8.4
# mlflow==2.13.0
# torch==2.3.0
# transformers==4.40.2
# xgboost==2.0.3
# lightgbm==4.3.0
# imbalanced-learn==0.12.2
# joblib==1.4.2
# Generate from your current environment:
# pip freeze > requirements.txtDocker Layer Caching for Fast Rebuilds
Docker builds each instruction as a separate cached layer. If a layer's inputs have not changed, Docker reuses the cached layer instead of re-executing the instruction. This means that placing COPY requirements.txt and RUN pip install before COPY . . (copying all code) ensures that changing code files does not trigger a slow re-installation of packages. Good Dockerfile order dramatically speeds up iterative development.
# Optimised Dockerfile layer order
# FROM python:3.11-slim
#
# WORKDIR /app
#
# # Step 1: Install system packages (rarely changes)
# RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*
#
# # Step 2: Install Python deps (changes only when requirements.txt changes)
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt
#
# # Step 3: Copy code (changes most often -- put last)
# COPY . .
#
# CMD ['python', 'train.py']
print('Layer order shown above.')GPU Support: CUDA Docker Images
For training deep learning models on GPU, use NVIDIA's official CUDA base images from Docker Hub. These images include the CUDA runtime and cuDNN libraries required by PyTorch and TensorFlow. The --gpus all flag when running the container exposes the host GPU to the container. The CUDA version in the image must match the drivers installed on the host machine.
# Dockerfile for GPU training with PyTorch
# FROM nvidia/cuda:12.1-cudnn8-runtime-ubuntu22.04
#
# # Install Python
# RUN apt-get update && apt-get install -y python3.11 python3-pip
# RUN ln -s /usr/bin/python3.11 /usr/bin/python
#
# WORKDIR /app
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt
# COPY train.py .
#
# CMD ['python', 'train.py']
# Run with GPU:
# docker run --rm --gpus all my-gpu-trainer:v1
print('GPU Dockerfile structure shown above.')Passing Configuration via Environment Variables
Avoid hard-coding hyperparameters in your training script. Instead, read them from environment variables with os.getenv and pass them via -e flags at container run time. This lets you launch the same image with different hyperparameters without rebuilding, making it easy to run hyperparameter sweeps by simply changing the run command.
import os
# In train.py -- read config from environment variables
LEARNING_RATE = float(os.getenv('LEARNING_RATE', '0.001'))
N_ESTIMATORS = int(os.getenv('N_ESTIMATORS', '100'))
MAX_DEPTH = int(os.getenv('MAX_DEPTH', '5'))
OUTPUT_DIR = os.getenv('OUTPUT_DIR', '/app/outputs')
print(f'Training with lr={LEARNING_RATE}, n={N_ESTIMATORS}, depth={MAX_DEPTH}')
# Run with custom config:
# docker run -e LEARNING_RATE=0.01 -e N_ESTIMATORS=200 my-ml-trainer:v1Mounting Volumes for Data and Outputs
Containers are ephemeral — data written inside a container disappears when it stops. Use volume mounts (-v flag) to bind a host directory into the container. Mount your data directory read-only and an output directory read-write. This keeps large datasets outside the image (reducing image size) and ensures model checkpoints and results persist after the container exits.
# Mount host data/ and outputs/ into container
# docker run --rm \
# -v /host/path/data:/app/data:ro \
# -v /host/path/outputs:/app/outputs \
# -e N_ESTIMATORS=200 \
# my-ml-trainer:v1
# In train.py, read from /app/data and write to /app/outputs
import os
data_dir = '/app/data'
output_dir = '/app/outputs'
os.makedirs(output_dir, exist_ok=True)
print('Data dir:', os.listdir(data_dir) if os.path.exists(data_dir) else 'not mounted')Multi-Stage Builds: Slim Production Images
Training images include compilers, header files, and dev tools that are unnecessary for inference. Multi-stage builds use one stage to compile/install everything and a second slim stage that copies only the final artifacts. This can reduce image size from 4 GB to under 200 MB, speeding up pulls and reducing the attack surface in production.
# Multi-stage Dockerfile
# --- Stage 1: Build ---
# FROM python:3.11 AS builder
# WORKDIR /app
# COPY requirements.txt .
# RUN pip install --no-cache-dir --user -r requirements.txt
#
# --- Stage 2: Runtime ---
# FROM python:3.11-slim
# WORKDIR /app
# COPY --from=builder /root/.local /root/.local
# COPY serve.py .
# COPY model/ model/
# ENV PATH=/root/.local/bin:$PATH
#
# CMD ['python', 'serve.py']
print('Multi-stage Dockerfile shown above.')Running MLflow Inside Docker
Combine Docker and MLflow by passing the MLflow tracking URI as an environment variable. The container trains the model, logs parameters and metrics to the remote MLflow server, and saves model artifacts to shared storage. This pattern is the building block of automated retraining pipelines: a scheduler triggers a Docker container that trains, evaluates, and registers a new model version without any manual intervention.
import os
import mlflow
from sklearn.ensemble import RandomForestClassifier
# Read from environment (set by docker run -e)
MLFLOW_URI = os.getenv('MLFLOW_TRACKING_URI', 'http://localhost:5000')
mlflow.set_tracking_uri(MLFLOW_URI)
mlflow.set_experiment('docker_training')
with mlflow.start_run():
n_est = int(os.getenv('N_ESTIMATORS', '100'))
mlflow.log_param('n_estimators', n_est)
clf = RandomForestClassifier(n_estimators=n_est, random_state=42)
# clf.fit(X_train, y_train) -- assume data is mounted
# acc = accuracy_score(y_test, clf.predict(X_test))
# mlflow.log_metric('accuracy', acc)
# mlflow.sklearn.log_model(clf, 'model')
print('Logged to:', MLFLOW_URI)Pushing Images to a Registry
To share Docker images with your team or deploy them to cloud infrastructure, push them to a container registry. Docker Hub is the public registry; AWS ECR, Google GCR, and Azure ACR are popular private alternatives. Tag your image with the registry URL and your image name, then push. CI/CD pipelines typically build a new image on every commit and push it with the commit SHA as the tag for full traceability.
# Tag and push to Docker Hub
# docker tag my-ml-trainer:v1 yourusername/my-ml-trainer:v1
# docker push yourusername/my-ml-trainer:v1
# Tag and push to AWS ECR
# aws ecr get-login-password --region eu-west-1 | \
# docker login --username AWS --password-stdin 123456789.dkr.ecr.eu-west-1.amazonaws.com
# docker tag my-ml-trainer:v1 123456789.dkr.ecr.eu-west-1.amazonaws.com/my-ml-trainer:v1
# docker push 123456789.dkr.ecr.eu-west-1.amazonaws.com/my-ml-trainer:v1
print('Registry push commands shown above.')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: Docker containers package the entire Python environment ensuring identical training results on any machine, layer caching makes builds fast by placing requirements installation before code copies, and volume mounts keep large datasets outside the image and persist outputs after the container exits. Next up we explore the MLflow Model Registry for promoting models through staging and production lifecycle stages.
자주 묻는 질문
“머신러닝을 위한 Docker 재현 가능 환경” 강의는 무료인가요?
네 — “머신러닝을 위한 Docker 재현 가능 환경” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Machine Learning Academy 강의 전체를 잠금 해제할 수 있습니다. Machine Learning Academy 강의에는 총 4개의 강의가 포함되어 있습니다.
“머신러닝을 위한 Docker 재현 가능 환경”에서 뭘 배우나요?
학습자는 Python, 버전이 고정된 머신러닝 라이브러리를 설치하고 학습 스크립트를 복사하는 Dockerfile을 작성한 뒤, 컨테이너를 빌드하고 실행하여 비트 단위의 재현 가능성을 확인합니다. 브라우저에서 직접 실행하는 실습 코드로 Machine Learning Academy을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.
Machine Learning Academy을(를) 시작하는 데 경험이 필요한가요?
사전 경험은 필요하지 않습니다. CoddyKit의 Machine Learning Academy은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 2번째 강의입니다.
“머신러닝을 위한 Docker 재현 가능 환경” 강의는 얼마나 걸리나요?
대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.
이 Machine Learning Academy 강의에서 코드를 작성하고 실행할 수 있나요?
네. 모든 Machine Learning Academy 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.
이 강의의 모든 강의
- MLflow를 활용한 실험 추적: 매개변수, 지표 및 산출물 기록
- 머신러닝을 위한 Docker 재현 가능 환경
- 모델 레지스트리: 스테이징, 운영 및 보관
- GitHub Actions를 활용한 자동 재학습 파이프라인