Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts
Learners will instrument a training script with mlflow.log_param and mlflow.log_metric, launch the MLflow UI, and compare runs across hyperparameter settings.
Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts is a free Machine Learning Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Machine Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Experiment Tracking Matters
When you train multiple models with different hyperparameters, it is easy to lose track of which run achieved the best result. Experiment tracking automatically records every training run's parameters, metrics, and code version in a searchable database. Without it, teams resort to messy spreadsheets or simply forget what worked. MLflow is the industry-standard open-source tool for tracking ML experiments and managing the model lifecycle.
Installing and Starting MLflow
MLflow is a pure Python package that requires no external database for getting started — it stores runs locally in an mlruns/ directory. Install with pip, then launch the tracking UI to visualise runs in a browser. The UI shows all experiments, each run's parameters and metrics, and lets you compare runs side-by-side with interactive charts.
# Install MLflow
# pip install mlflow scikit-learn
import mlflow
import mlflow.sklearn
# Start the tracking UI (run this in a terminal):
# mlflow ui --host 0.0.0.0 --port 5000
# Then open http://localhost:5000
# Check MLflow version
print('MLflow version:', mlflow.__version__)
# Default tracking URI stores to ./mlruns
print('Tracking URI:', mlflow.get_tracking_uri())Creating Experiments
MLflow organises runs into experiments — logical groupings of related runs. Create a named experiment with mlflow.set_experiment. All subsequent runs belong to this experiment. Use descriptive names that identify the project and date, e.g., 'random_forest_imdb_2024'. Each run within an experiment gets a unique ID, a human-readable name, and its own parameter/metric/artifact store.
import mlflow
# Create or switch to an experiment
experiment_name = 'sentiment_classification'
mlflow.set_experiment(experiment_name)
# List all experiments
for exp in mlflow.search_experiments():
print(f'ID: {exp.experiment_id} Name: {exp.name}')
# Get current experiment info
experiment = mlflow.get_experiment_by_name(experiment_name)
print('Artifact location:', experiment.artifact_location)Logging Parameters with mlflow.log_param
Parameters are the configuration choices made before training begins: hyperparameters like learning rate, number of estimators, and regularisation strength. Log them with mlflow.log_param(key, value) inside an active run context. Parameters are immutable once logged — they define the experiment setup and help you filter runs to find the configuration that worked best.
import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
with mlflow.start_run(run_name='rf_baseline'):
n_estimators = 100
max_depth = 5
mlflow.log_param('n_estimators', n_estimators)
mlflow.log_param('max_depth', max_depth)
mlflow.log_param('random_state', 42)
clf = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth, random_state=42)
clf.fit(X_train, y_train)
print('Params logged.')Logging Metrics with mlflow.log_metric
Metrics are the quantitative results produced during and after training: accuracy, F1, loss values. Log them with mlflow.log_metric(key, value) or mlflow.log_metric(key, value, step=i) to track progress across epochs. Logging metrics at each epoch lets MLflow plot learning curves in the UI, making it easy to compare convergence speed across runs.
import mlflow
from sklearn.metrics import accuracy_score, f1_score
with mlflow.start_run(run_name='rf_run_1'):
mlflow.log_param('n_estimators', 100)
mlflow.log_param('max_depth', 5)
# Train model (abbreviated)
clf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
clf.fit(X_train, y_train)
preds = clf.predict(X_test)
acc = accuracy_score(y_test, preds)
f1 = f1_score(y_test, preds)
mlflow.log_metric('test_accuracy', acc)
mlflow.log_metric('test_f1', f1)
print(f'Logged accuracy={acc:.4f}, f1={f1:.4f}')Logging Artifacts
Artifacts are files associated with a run: trained model weights, plots, confusion matrices, or feature importance charts. Log them with mlflow.log_artifact(local_path). MLflow stores the file in the run's artifact directory and makes it downloadable from the UI. Logging the confusion matrix image alongside metrics gives reviewers a complete picture without diving into code.
import mlflow
import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay
import numpy as np
with mlflow.start_run():
clf.fit(X_train, y_train)
preds = clf.predict(X_test)
# Save confusion matrix as an artifact
fig, ax = plt.subplots(figsize=(5, 4))
ConfusionMatrixDisplay.from_predictions(y_test, preds, ax=ax)
plt.tight_layout()
plt.savefig('/tmp/confusion_matrix.png')
plt.close()
mlflow.log_artifact('/tmp/confusion_matrix.png')
print('Artifact logged.')Auto-logging with mlflow.sklearn.autolog
For scikit-learn, MLflow provides auto-logging that captures all parameters, metrics, and the trained model automatically without any manual log calls. Simply call mlflow.sklearn.autolog() before fitting. Auto-logging supports XGBoost, LightGBM, PyTorch, TensorFlow, and more with similar one-line activation. It is the fastest way to start tracking without modifying existing training code.
import mlflow
import mlflow.sklearn
from sklearn.ensemble import GradientBoostingClassifier
# Enable auto-logging -- no manual log calls needed
mlflow.sklearn.autolog()
with mlflow.start_run(run_name='gb_autolog'):
clf = GradientBoostingClassifier(n_estimators=200, max_depth=3, learning_rate=0.05)
clf.fit(X_train, y_train)
# MLflow automatically logs:
# - All constructor params
# - Training accuracy
# - Model artifact
print('Auto-logged run complete.')Comparing Runs in the MLflow UI
After several runs with different hyperparameters, open the MLflow UI at http://localhost:5000. Select multiple runs with checkboxes and click Compare to view them side by side. The comparison page shows a parallel coordinates plot where each axis is a parameter or metric, making it easy to spot which parameter combination led to the highest accuracy. You can also sort the run table by any metric column.
import mlflow
# Search runs programmatically (useful in CI/CD pipelines)
runs = mlflow.search_runs(
experiment_names=['sentiment_classification'],
order_by=['metrics.test_accuracy DESC']
)
if not runs.empty:
best_run = runs.iloc[0]
print('Best run ID:', best_run['run_id'])
print('Best accuracy:', best_run['metrics.test_accuracy'])
print('n_estimators:', best_run['params.n_estimators'])
print('max_depth:', best_run['params.max_depth'])Logging Models with mlflow.sklearn.log_model
Beyond logging model files as generic artifacts, use mlflow.sklearn.log_model to log the model in MLflow's standardised format. This format includes the model object, its flavour metadata, and an automatically generated MLmodel config file. The standardised format enables one-click deployment to local REST servers (mlflow models serve) and cloud platforms like Azure ML and AWS SageMaker.
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
with mlflow.start_run(run_name='rf_logged_model'):
clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_train, y_train)
mlflow.log_metric('accuracy', accuracy_score(y_test, clf.predict(X_test)))
# Log in sklearn flavour -- enables mlflow models serve
mlflow.sklearn.log_model(
sk_model=clf,
artifact_path='random_forest',
registered_model_name='SentimentRF'
)
print('Model registered in Model Registry.')Tagging Runs for Organisation
MLflow runs can have tags: key-value metadata that goes beyond structured parameters and metrics. Use tags to store the dataset version, code commit hash, experimenter name, or notes about what changed. Tags are searchable and filterable in the UI. A good tagging convention — like always logging dataset_version and git_sha — ensures every run is reproducible and auditable months later.
import mlflow
import subprocess
def get_git_sha():
try:
return subprocess.check_output(
['git', 'rev-parse', 'HEAD'], text=True
).strip()
except Exception:
return 'unknown'
with mlflow.start_run():
mlflow.set_tag('dataset_version', 'imdb_v2')
mlflow.set_tag('git_sha', get_git_sha())
mlflow.set_tag('author', 'mehmet.canker')
mlflow.set_tag('notes', 'Testing higher max_depth after previous plateau')
mlflow.log_param('max_depth', 8)
print('Run tagged.')Remote Tracking Server Setup
For team collaboration, point MLflow at a remote tracking server instead of the local mlruns/ folder. Set the tracking URI to your server's URL before any MLflow calls. All team members log to the same server, see each other's runs, and access shared artifacts. The server backend can use a PostgreSQL database for metadata and S3/GCS for artifact storage, providing enterprise-grade durability and access control.
import mlflow
import os
# Point to a remote MLflow tracking server
os.environ['MLFLOW_TRACKING_URI'] = 'http://mlflow.yourcompany.com:5000'
os.environ['MLFLOW_TRACKING_USERNAME'] = 'data_team'
os.environ['MLFLOW_TRACKING_PASSWORD'] = 'secret'
mlflow.set_tracking_uri(os.environ['MLFLOW_TRACKING_URI'])
# All subsequent mlflow calls go to the remote server
mlflow.set_experiment('shared_team_experiment')
with mlflow.start_run():
mlflow.log_param('model_type', 'xgboost')
mlflow.log_metric('auc', 0.94)
print('Logged to remote server.')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: MLflow tracks parameters, metrics, and artifacts for every training run in a searchable database, mlflow.sklearn.autolog() captures everything automatically without manual log calls, and logged models in the sklearn flavour can be served as REST APIs or promoted through the Model Registry. Next up we build reproducible Docker containers for ML training to eliminate environment differences between machines.
Frequently asked questions
Is the “Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts” lesson free?
Yes — the full text of “Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts” is free to read here on the web, and the Machine Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Machine Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts”?
Learners will instrument a training script with mlflow.log_param and mlflow.log_metric, launch the MLflow UI, and compare runs across hyperparameter settings. You practise Machine Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Machine Learning Academy?
No prior experience is required. Machine Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Machine Learning Academy lesson?
Yes. Every Machine Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Experiment Tracking with MLflow: Log Params, Metrics, and Artifacts
- Reproducible Environments with Docker for ML
- Model Registry: Staging, Production, and Archiving
- Automated Retraining Pipelines with GitHub Actions