The ML Workflow: Data to Prediction
Learners will walk through the end-to-end pipeline from raw data collection and cleaning to model training, evaluation, and deployment.
The ML Workflow: Data to Prediction is a free Machine Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Machine Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The End-to-End ML Pipeline
Building ML is more than training a model — it's a full pipeline. Knowing all the stages keeps you from rushing straight to modelling too soon.
Stage 1: Define the Problem
Stage one is to define the problem: what are you predicting, why does it matter, and how will you measure success? Vague goals make vague models.
Stage 2: Collect and Load Data
Next, collect and load your data from files, databases, or APIs — Pandas is the go-to tool. Always inspect the raw data before doing anything else.
import pandas as pd
# Load data from a CSV file
df = pd.read_csv('housing.csv')
# First inspection
print(df.shape) # (rows, columns)
print(df.dtypes) # data types per column
print(df.head()) # first 5 rows
print(df.describe()) # summary statisticsStage 3: Exploratory Data Analysis
Exploratory Data Analysis is the detective work: plot distributions, check correlations, and hunt for outliers before you build any model.
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
df = pd.read_csv('housing.csv')
# Distribution of target variable
df['price'].hist(bins=50)
plt.title('House Price Distribution')
plt.show()
# Correlation heat map
sns.heatmap(df.corr(), annot=True, cmap='coolwarm')
plt.show()
# Check for missing values
print(df.isnull().sum())Stage 4: Preprocess the Data
Preprocessing turns messy data into a clean feature matrix — filling gaps, scaling, and encoding. Golden rule: fit only on training data to avoid leakage.
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import pandas as pd
import numpy as np
df = pd.read_csv('housing.csv')
X = df.drop('price', axis=1)
y = df['price']
# Split first, then fit scaler ONLY on train
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # fit + transform on train
X_test_scaled = scaler.transform(X_test) # transform only on testStage 5: Train the Model
Now train the model: in scikit-learn it's one fit() call. Start simple with a baseline like a linear model before reaching for anything complex.
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
# Simple baseline first
linear_model = LinearRegression()
linear_model.fit(X_train_scaled, y_train)
# More complex model
rf_model = RandomForestRegressor(n_estimators=100, random_state=42)
rf_model.fit(X_train_scaled, y_train)
print('Linear model trained.')
print('Random Forest trained.')Stage 6: Evaluate the Model
Evaluate on held-out test data the model never saw. Pick the right metric — MAE and RMSE for numbers, accuracy and F1 for categories.
from sklearn.metrics import mean_absolute_error, r2_score
import numpy as np
y_pred = linear_model.predict(X_test_scaled)
mae = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
rmse = np.sqrt(((y_test - y_pred) ** 2).mean())
print(f'MAE: {mae:.2f}')
print(f'RMSE: {rmse:.2f}')
print(f'R2: {r2:.3f}')Stage 7: Iterate and Improve
The first model is rarely the best. ML is iterative: based on your results, gather more data, engineer features, or tune settings — then try again.
Stage 8: Deploy the Model
Deployment makes your trained model available to real users — often as a web API, a batch job, or an on-device model in an app. A notebook alone adds no value.
import joblib
# Save the trained model
joblib.dump(linear_model, 'house_price_model.pkl')
print('Model saved.')
# Later, load and predict in production
loaded_model = joblib.load('house_price_model.pkl')
prediction = loaded_model.predict(X_test_scaled[:1])
print(f'Prediction: ${prediction[0]:,.0f}')Stage 9: Monitor and Retrain
Deployment isn't the finish line. Data shifts over time, so you need monitoring to catch silent performance drops and trigger retraining when needed.
The Workflow as a Scikit-learn Pipeline
A scikit-learn Pipeline chains preprocessing and modelling into one object. It prevents leakage and makes deployment cleaner — save and load just one thing.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
# Chain preprocessing and model in one object
pipeline = Pipeline([
('scaler', StandardScaler()),
('model', LinearRegression())
])
# fit() applies scaler then trains the model
pipeline.fit(X_train, y_train)
# predict() applies scaler then predicts
y_pred = pipeline.predict(X_test)
print('Pipeline prediction done.')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
Great progress! The ML workflow runs from problem definition to monitoring, preprocessing fits on training data only, and deployment is the start, not the end.
Frequently asked questions
Is the “The ML Workflow: Data to Prediction” lesson free?
Yes — the full text of “The ML Workflow: Data to Prediction” is free to read here on the web, and the Machine Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Machine Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “The ML Workflow: Data to Prediction”?
Learners will walk through the end-to-end pipeline from raw data collection and cleaning to model training, evaluation, and deployment. You practise Machine Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Machine Learning Academy?
No prior experience is required. Machine Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “The ML Workflow: Data to Prediction” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Machine Learning Academy lesson?
Yes. Every Machine Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.