0Pricing
Machine Learning Academy · Pelajaran

Menggabungkan Langkah dengan ColumnTransformer

Peserta didik akan menerapkan langkah prapemrosesan yang berbeda pada berbagai jenis kolom secara bersamaan menggunakan ColumnTransformer, sehingga menghasilkan satu matriks fitur yang bersih.

Menggabungkan Langkah dengan ColumnTransformer adalah pelajaran Machine Learning Academy gratis di CoddyKit. Ini adalah pelajaran 4 dari 4. Kamu bisa membaca pelajaran lengkapnya di bawah secara gratis — lalu praktikkan langsung di browser dengan editor kode bawaan dan tutor AI 24/7. Ini adalah bagian dari jalur belajar Machine Learning Academy, dan progresmu tersinkronisasi di web dan aplikasi CoddyKit. Kursus Machine Learning Academy mencakup 4 pelajaran total.

Bagian dari pelajaran ini belum diterjemahkan dan ditampilkan dalam bahasa Inggris.

The Problem ColumnTransformer Solves

Real datasets contain a mix of column types: some are numeric, some are ordinal, and some are nominal categorical. Applying the same transformer to all columns would be incorrect — you cannot StandardScale a one-hot column or OrdinalEncode a continuous number. Before ColumnTransformer, data scientists had to manually slice DataFrames, apply different transformers, and re-concatenate — a fragile, leakage-prone process. ColumnTransformer solves this by applying different transformations to different column subsets simultaneously inside a single, pipeline-compatible object.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'age':       [25, 35, 45, 28],
    'income':    [40000, 80000, 120000, 55000],
    'education': ['bachelor', 'master', 'PhD', 'high school'],
    'city':      ['NYC', 'LA', 'NYC', 'Chicago']
})

# Need: StandardScaler for age+income
#       OrdinalEncoder for education
#       OneHotEncoder for city
# ColumnTransformer applies all three at once

ColumnTransformer Syntax and Structure

A ColumnTransformer is constructed as a list of (name, transformer, columns) tuples. The name is a string identifier used in logging and parameter access. The transformer is any scikit-learn estimator with fit and transform. The columns can be a list of column names (for DataFrames), integer indices (for arrays), or a boolean mask. All transformations are applied in parallel, and outputs are concatenated column-wise into a single matrix.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ordinal', OrdinalEncoder(
        categories=[['high school', 'bachelor', 'master', 'PhD']]
    ), ['education']),
    ('ohe', OneHotEncoder(drop='first', sparse_output=False), ['city'])
])

# Output: scaled numeric + ordinal int + one-hot columns concatenated

Fitting and Transforming Data

Like any scikit-learn transformer, ColumnTransformer has fit(), transform(), and fit_transform() methods. When you call fit(X_train), every sub-transformer is fitted on the training data simultaneously. When you call transform(X_test), each transformer applies its learnt parameters (e.g., scaler mean, encoder categories) to the test data. Each sub-transformer only touches its designated columns, so the fit statistics are clean and isolated.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.model_selection import train_test_split
import pandas as pd

X_train, X_test = train_test_split(df, test_size=0.2, random_state=42)

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ohe', OneHotEncoder(sparse_output=False), ['city'])
])

ct.fit(X_train)  # Learn stats from training only
X_train_t = ct.transform(X_train)
X_test_t  = ct.transform(X_test)  # Apply training stats to test

print('Transformed train shape:', X_train_t.shape)

The remainder Parameter

By default, any column not explicitly listed in ColumnTransformer is dropped from the output. The remainder parameter controls this behaviour. Set remainder='passthrough' to keep unspecified columns unchanged (appended at the end of the output matrix). Set remainder=StandardScaler() to apply a transformer to all columns not mentioned. Using 'passthrough' is convenient when most columns are numeric and you only need to specify special treatment for a few categorical ones.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

# Only encode 'city'; pass all other columns through unchanged
ct = ColumnTransformer(
    transformers=[
        ('ohe', OneHotEncoder(sparse_output=False), ['city'])
    ],
    remainder='passthrough'  # age, income, education appended as-is
)

X_transformed = ct.fit_transform(df)
print('Shape with passthrough:', X_transformed.shape)

Getting Feature Names from ColumnTransformer

After transformation, the output matrix column count differs from the input because one-hot encoding adds columns. Call ct.get_feature_names_out() to retrieve the names of all output columns. Names are prefixed by the transformer name (e.g., ohe__city_NYC, scale__age). This is essential for interpreting feature importances, understanding SHAP values, or debugging when model performance is unexpectedly poor on specific features.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ohe', OneHotEncoder(sparse_output=False), ['city'])
])

ct.fit(df[['age', 'income', 'city']])
feature_names = ct.get_feature_names_out()
print('Output feature names:', feature_names)

Column Specification Methods

ColumnTransformer supports multiple ways to specify which columns to transform: by name (string list — most readable), by index (integer list — for arrays without column names), or with a boolean selector using make_column_selector() to automatically detect numeric or categorical columns. The make_column_selector(dtype_include=np.number) pattern is especially powerful for DataFrames where you want to automatically scale all numeric columns without listing them individually.

from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.preprocessing import StandardScaler, OneHotEncoder
import numpy as np

# Automatically select numeric vs categorical columns
ct = ColumnTransformer([
    ('scale', StandardScaler(),
     make_column_selector(dtype_include=np.number)),
    ('ohe', OneHotEncoder(sparse_output=False),
     make_column_selector(dtype_include=object))
])

X_out = ct.fit_transform(df)
print('Auto-detected shape:', X_out.shape)

Nesting ColumnTransformer in a Pipeline

The real power of ColumnTransformer emerges when it is nested inside a Pipeline. The ColumnTransformer handles preprocessing, and the downstream estimator handles learning. Wrapping both in a pipeline lets you pass raw DataFrames directly to fit() and predict(). When used with cross_val_score, the pipeline ensures that preprocessing is refitted on each training fold — preventing any test data from influencing transformation parameters.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), ['age', 'income']),
    ('cat', OneHotEncoder(handle_unknown='ignore'), ['city', 'education'])
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('clf', RandomForestClassifier(n_estimators=100, random_state=42))
])

scores = cross_val_score(pipeline, X, y, cv=5)
print('CV mean accuracy:', scores.mean().round(3))

Grid Searching Pipeline Parameters

When your ColumnTransformer is inside a Pipeline, you can tune the hyperparameters of any step using GridSearchCV. Access nested parameters with double-underscore notation: prep__num__with_mean targets the with_mean parameter of the num transformer inside prep (the ColumnTransformer). Similarly, clf__n_estimators tunes the classifier. This makes end-to-end hyperparameter optimisation possible without manually re-running preprocessing.

from sklearn.model_selection import GridSearchCV

param_grid = {
    'prep__num__with_mean': [True, False],  # StandardScaler param
    'clf__n_estimators': [50, 100, 200],
    'clf__max_depth': [None, 5, 10]
}

grid = GridSearchCV(pipeline, param_grid, cv=5, scoring='accuracy')
grid.fit(X_train, y_train)

print('Best params:', grid.best_params_)
print('Best CV score:', grid.best_score_.round(3))

Handling Mixed Imputation and Encoding

A complete preprocessing pipeline often chains imputation and encoding for the same column type. Scikit-learn supports nested pipelines: you can pass a Pipeline as the transformer inside a ColumnTransformer. For example, numeric columns might need imputation followed by scaling, while categorical columns need imputation followed by one-hot encoding. This eliminates any manual intermediate steps and keeps the entire transformation reproducible in a single fit call.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

num_pipe = Pipeline([
    ('impute', SimpleImputer(strategy='median')),
    ('scale', StandardScaler())
])

cat_pipe = Pipeline([
    ('impute', SimpleImputer(strategy='most_frequent')),
    ('ohe', OneHotEncoder(handle_unknown='ignore', sparse_output=False))
])

preprocessor = ColumnTransformer([
    ('num', num_pipe, ['age', 'income']),
    ('cat', cat_pipe, ['city', 'education'])
])

Inspecting Sub-Transformers

After fitting a ColumnTransformer, you can inspect each sub-transformer by name to debug or extract learned parameters. Use ct.named_transformers_ to access transformers by their string name. For a nested pipeline inside ColumnTransformer, chain attribute access: ct.named_transformers_['num']['scale'].mean_ retrieves the learned mean from the StandardScaler inside the numeric sub-pipeline. This introspection is invaluable for confirming that preprocessing ran correctly on the right columns.

# After fitting the preprocessor:
preprocessor.fit(X_train)

# Access the numeric sub-pipeline
num_pipeline = preprocessor.named_transformers_['num']
print('Imputer strategy:', num_pipeline['impute'].strategy)
print('Scaler mean:', num_pipeline['scale'].mean_)

# Access the OHE categories
cat_pipeline = preprocessor.named_transformers_['cat']
print('OHE categories:', cat_pipeline['ohe'].categories_)

Persisting and Reloading a Fitted ColumnTransformer

A fitted ColumnTransformer (or the full pipeline containing it) stores all learned parameters: scaling means and standard deviations, encoder category lists, imputer fill values. Serialise it with joblib.dump() and reload it later without re-training. This is how preprocessing is deployed to production: the fitted transformer ensures that incoming data is processed identically to training data, so predictions are consistent and correct across sessions.

import joblib

# Save the fitted pipeline
joblib.dump(pipeline, 'model_pipeline.joblib')

# Load and use in production without re-training
loaded_pipeline = joblib.load('model_pipeline.joblib')

# New raw data in the same format as training data
X_new = pd.DataFrame({
    'age': [30], 'income': [65000],
    'city': ['NYC'], 'education': ['master']
})

prediction = loaded_pipeline.predict(X_new)
print('Prediction:', prediction)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: how ColumnTransformer applies different transformations to different column subsets simultaneously, how to nest it inside a Pipeline for leak-free cross-validation, and how to compose imputation and encoding together using sub-pipelines. Next up we explore the K-Nearest Neighbors algorithm and how it classifies new data points using distance.

Pertanyaan yang Sering Diajukan

Apakah pelajaran “Menggabungkan Langkah dengan ColumnTransformer” gratis?

Ya — teks lengkap “Menggabungkan Langkah dengan ColumnTransformer” gratis dibaca di sini di web. Untuk praktiknya secara interaktif (editor kode bawaan dan tutor AI 24/7) dan buka sisa kursus Machine Learning Academy, upgrade ke CoddyKit PRO. Kursus Machine Learning Academy mencakup 4 pelajaran total.

Apa yang akan aku pelajari di “Menggabungkan Langkah dengan ColumnTransformer”?

Peserta didik akan menerapkan langkah prapemrosesan yang berbeda pada berbagai jenis kolom secara bersamaan menggunakan ColumnTransformer, sehingga menghasilkan satu matriks fitur yang bersih. Kamu berlatih Machine Learning Academy dengan kode praktik yang langsung kamu jalankan di browser, dan tutor AI 24/7 menjawab pertanyaanmu saat kamu mengerjakan pelajaran ini.

Apakah aku perlu pengalaman untuk memulai Machine Learning Academy?

Tidak diperlukan pengalaman sebelumnya. Machine Learning Academy di CoddyKit dirancang untuk pemula hingga pelajar tingkat lanjut, jadi kamu bisa memulai di sini atau dari awal dan belajar sesuai kecepatan kamu sendiri. Ini adalah pelajaran 4 dari 4.

Berapa lama pelajaran “Menggabungkan Langkah dengan ColumnTransformer” memakan waktu?

Sebagian besar pelajaran CoddyKit memakan waktu sekitar 5–10 menit. Setiap pelajaran ringkas dan interaktif, jadi kamu membuat kemajuan stabil dan melanjutkan dari tempat kamu tinggalkan di web dan aplikasi.

Bisakah aku menulis dan menjalankan kode dalam pelajaran Machine Learning Academy ini?

Ya. Setiap pelajaran Machine Learning Academy menyertakan editor kode bawaan, jadi kamu menulis dan menjalankan kode nyata langsung di browser dan mendapatkan umpan balik AI instan — tidak diperlukan penyiapan lokal.

Semua pelajaran dalam kursus ini

  1. Menangani Nilai yang Hilang: Hapus, Imputasi, dan Tandai
  2. Penskalaan Fitur: StandardScaler dan MinMaxScaler
  3. Pengodean Variabel Kategorikal: OrdinalEncoder dan OneHotEncoder
  4. Menggabungkan Langkah dengan ColumnTransformer
← Kembali ke Machine Learning Academy