Building Preprocessing Pipelines
sklearn Pipeline, ColumnTransformer, combining transformers for clean preprocessing workflows.
Building Preprocessing Pipelines is a free Learn AI with Python lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Pipelines?
A scikit-learn Pipeline chains preprocessing steps and a model into one object. It guarantees the same transformations apply to train and test data, preventing leakage and messy code.
A Basic Pipeline
Pipeline takes a list of (name, step) tuples. Calling fit runs each step in order; the final step is usually an estimator.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipe = Pipeline([
("scale", StandardScaler()),
("model", LogisticRegression()),
])fit and predict the Whole Pipeline
Treat the pipeline like a single model. fit learns the scaler AND trains the model; predict applies the scaler then the model, automatically and consistently.
pipe.fit(X_train, y_train)
preds = pipe.predict(X_test) # scaling applied automaticallyNo Leakage by Design
Because the pipeline fits the scaler only during fit (on training data) and merely transforms during predict, it eliminates the fit-on-test mistake automatically.
Mixed Column Types
Real datasets mix NUMERIC and CATEGORICAL columns that need different preprocessing. ColumnTransformer applies a different transformer to each subset of columns.
ColumnTransformer
Provide tuples of (name, transformer, columns). Numeric columns get scaled; categorical columns get one-hot encoded, all in one step.
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
pre = ColumnTransformer([
("num", StandardScaler(), ["age", "income"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])Nesting in a Full Pipeline
Put the ColumnTransformer as the first step of a Pipeline, followed by the model, for a complete, leak-free workflow.
full = Pipeline([
("prep", pre),
("model", LogisticRegression()),
])
full.fit(X_train, y_train)Handling Missing Values in a Step
Add an SimpleImputer inside the numeric branch (often itself a small pipeline) to fill missing values before scaling, keeping everything inside the pipeline.
from sklearn.impute import SimpleImputer
num_pipe = Pipeline([
("impute", SimpleImputer(strategy="median")),
("scale", StandardScaler()),
])fit_transform on Train, transform on Test
The same golden rule applies to the whole pipeline: it learns every parameter from training data during fit, then only transforms test data during predict or transform.
full.fit(X_train, y_train) # learns imputer, scaler, encoder, model
full.predict(X_test) # transform-only pathPersisting a Pipeline
Save the entire fitted pipeline, preprocessing plus model, to disk with joblib. Loading it later applies identical preprocessing in production, no manual steps to reproduce.
import joblib
joblib.dump(full, "model.joblib")
loaded = joblib.load("model.joblib")
loaded.predict(X_new) # same preprocessing guaranteedWhy This Matters in Production
A persisted pipeline means the deployed model preprocesses incoming data EXACTLY as during training. This consistency is the single biggest defense against train/serve skew bugs.
Quick Check
Test your pipeline knowledge.
Recap
Pipeline toolkit:
Pipelinechains preprocessing + model into one fit/predict object- No leakage: parameters learned on
fit, applied onpredict ColumnTransformerhandles mixed numeric/categorical columnsSimpleImputerfor missing values inside the pipeline- Persist with
joblibfor consistent production preprocessing
Frequently asked questions
Is the “Building Preprocessing Pipelines” lesson free?
Yes — the full text of “Building Preprocessing Pipelines” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Building Preprocessing Pipelines”?
sklearn Pipeline, ColumnTransformer, combining transformers for clean preprocessing workflows. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Building Preprocessing Pipelines” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Outlier Detection and Removal
- Encoding Categorical Variables
- Feature Scaling: Normalization and Standardization
- Building Preprocessing Pipelines