0Pricing
Machine Learning Academy · Lektion

Schritte mit ColumnTransformer kombinieren

Wenden Sie mit ColumnTransformer gleichzeitig unterschiedliche Vorverarbeitungsschritte auf verschiedene Spaltentypen an und erzeugen Sie eine einheitliche, bereinigte Feature-Matrix.

Schritte mit ColumnTransformer kombinieren ist eine kostenlose Machine Learning Academy-Lektion auf CoddyKit. Dies ist Lektion 4 von 4. Du kannst die komplette Lektion unten kostenlos lesen – dann übst du sie direkt im Browser mit einem integrierten Code-Editor und einem KI-Tutor rund um die Uhr. Sie ist Teil des Machine Learning Academy-Lernpfads, und dein Fortschritt wird über Web und CoddyKit-App synchronisiert. Der Machine Learning Academy-Kurs umfasst insgesamt 4 Lektionen.

Teile dieser Lektion wurden noch nicht übersetzt und werden auf Englisch angezeigt.

The Problem ColumnTransformer Solves

Real datasets contain a mix of column types: some are numeric, some are ordinal, and some are nominal categorical. Applying the same transformer to all columns would be incorrect — you cannot StandardScale a one-hot column or OrdinalEncode a continuous number. Before ColumnTransformer, data scientists had to manually slice DataFrames, apply different transformers, and re-concatenate — a fragile, leakage-prone process. ColumnTransformer solves this by applying different transformations to different column subsets simultaneously inside a single, pipeline-compatible object.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'age':       [25, 35, 45, 28],
    'income':    [40000, 80000, 120000, 55000],
    'education': ['bachelor', 'master', 'PhD', 'high school'],
    'city':      ['NYC', 'LA', 'NYC', 'Chicago']
})

# Need: StandardScaler for age+income
#       OrdinalEncoder for education
#       OneHotEncoder for city
# ColumnTransformer applies all three at once

ColumnTransformer Syntax and Structure

A ColumnTransformer is constructed as a list of (name, transformer, columns) tuples. The name is a string identifier used in logging and parameter access. The transformer is any scikit-learn estimator with fit and transform. The columns can be a list of column names (for DataFrames), integer indices (for arrays), or a boolean mask. All transformations are applied in parallel, and outputs are concatenated column-wise into a single matrix.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ordinal', OrdinalEncoder(
        categories=[['high school', 'bachelor', 'master', 'PhD']]
    ), ['education']),
    ('ohe', OneHotEncoder(drop='first', sparse_output=False), ['city'])
])

# Output: scaled numeric + ordinal int + one-hot columns concatenated

Fitting and Transforming Data

Like any scikit-learn transformer, ColumnTransformer has fit(), transform(), and fit_transform() methods. When you call fit(X_train), every sub-transformer is fitted on the training data simultaneously. When you call transform(X_test), each transformer applies its learnt parameters (e.g., scaler mean, encoder categories) to the test data. Each sub-transformer only touches its designated columns, so the fit statistics are clean and isolated.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.model_selection import train_test_split
import pandas as pd

X_train, X_test = train_test_split(df, test_size=0.2, random_state=42)

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ohe', OneHotEncoder(sparse_output=False), ['city'])
])

ct.fit(X_train)  # Learn stats from training only
X_train_t = ct.transform(X_train)
X_test_t  = ct.transform(X_test)  # Apply training stats to test

print('Transformed train shape:', X_train_t.shape)

The remainder Parameter

By default, any column not explicitly listed in ColumnTransformer is dropped from the output. The remainder parameter controls this behaviour. Set remainder='passthrough' to keep unspecified columns unchanged (appended at the end of the output matrix). Set remainder=StandardScaler() to apply a transformer to all columns not mentioned. Using 'passthrough' is convenient when most columns are numeric and you only need to specify special treatment for a few categorical ones.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

# Only encode 'city'; pass all other columns through unchanged
ct = ColumnTransformer(
    transformers=[
        ('ohe', OneHotEncoder(sparse_output=False), ['city'])
    ],
    remainder='passthrough'  # age, income, education appended as-is
)

X_transformed = ct.fit_transform(df)
print('Shape with passthrough:', X_transformed.shape)

Getting Feature Names from ColumnTransformer

After transformation, the output matrix column count differs from the input because one-hot encoding adds columns. Call ct.get_feature_names_out() to retrieve the names of all output columns. Names are prefixed by the transformer name (e.g., ohe__city_NYC, scale__age). This is essential for interpreting feature importances, understanding SHAP values, or debugging when model performance is unexpectedly poor on specific features.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

ct = ColumnTransformer([
    ('scale', StandardScaler(), ['age', 'income']),
    ('ohe', OneHotEncoder(sparse_output=False), ['city'])
])

ct.fit(df[['age', 'income', 'city']])
feature_names = ct.get_feature_names_out()
print('Output feature names:', feature_names)

Column Specification Methods

ColumnTransformer supports multiple ways to specify which columns to transform: by name (string list — most readable), by index (integer list — for arrays without column names), or with a boolean selector using make_column_selector() to automatically detect numeric or categorical columns. The make_column_selector(dtype_include=np.number) pattern is especially powerful for DataFrames where you want to automatically scale all numeric columns without listing them individually.

from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.preprocessing import StandardScaler, OneHotEncoder
import numpy as np

# Automatically select numeric vs categorical columns
ct = ColumnTransformer([
    ('scale', StandardScaler(),
     make_column_selector(dtype_include=np.number)),
    ('ohe', OneHotEncoder(sparse_output=False),
     make_column_selector(dtype_include=object))
])

X_out = ct.fit_transform(df)
print('Auto-detected shape:', X_out.shape)

Nesting ColumnTransformer in a Pipeline

The real power of ColumnTransformer emerges when it is nested inside a Pipeline. The ColumnTransformer handles preprocessing, and the downstream estimator handles learning. Wrapping both in a pipeline lets you pass raw DataFrames directly to fit() and predict(). When used with cross_val_score, the pipeline ensures that preprocessing is refitted on each training fold — preventing any test data from influencing transformation parameters.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), ['age', 'income']),
    ('cat', OneHotEncoder(handle_unknown='ignore'), ['city', 'education'])
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('clf', RandomForestClassifier(n_estimators=100, random_state=42))
])

scores = cross_val_score(pipeline, X, y, cv=5)
print('CV mean accuracy:', scores.mean().round(3))

Grid Searching Pipeline Parameters

When your ColumnTransformer is inside a Pipeline, you can tune the hyperparameters of any step using GridSearchCV. Access nested parameters with double-underscore notation: prep__num__with_mean targets the with_mean parameter of the num transformer inside prep (the ColumnTransformer). Similarly, clf__n_estimators tunes the classifier. This makes end-to-end hyperparameter optimisation possible without manually re-running preprocessing.

from sklearn.model_selection import GridSearchCV

param_grid = {
    'prep__num__with_mean': [True, False],  # StandardScaler param
    'clf__n_estimators': [50, 100, 200],
    'clf__max_depth': [None, 5, 10]
}

grid = GridSearchCV(pipeline, param_grid, cv=5, scoring='accuracy')
grid.fit(X_train, y_train)

print('Best params:', grid.best_params_)
print('Best CV score:', grid.best_score_.round(3))

Handling Mixed Imputation and Encoding

A complete preprocessing pipeline often chains imputation and encoding for the same column type. Scikit-learn supports nested pipelines: you can pass a Pipeline as the transformer inside a ColumnTransformer. For example, numeric columns might need imputation followed by scaling, while categorical columns need imputation followed by one-hot encoding. This eliminates any manual intermediate steps and keeps the entire transformation reproducible in a single fit call.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

num_pipe = Pipeline([
    ('impute', SimpleImputer(strategy='median')),
    ('scale', StandardScaler())
])

cat_pipe = Pipeline([
    ('impute', SimpleImputer(strategy='most_frequent')),
    ('ohe', OneHotEncoder(handle_unknown='ignore', sparse_output=False))
])

preprocessor = ColumnTransformer([
    ('num', num_pipe, ['age', 'income']),
    ('cat', cat_pipe, ['city', 'education'])
])

Inspecting Sub-Transformers

After fitting a ColumnTransformer, you can inspect each sub-transformer by name to debug or extract learned parameters. Use ct.named_transformers_ to access transformers by their string name. For a nested pipeline inside ColumnTransformer, chain attribute access: ct.named_transformers_['num']['scale'].mean_ retrieves the learned mean from the StandardScaler inside the numeric sub-pipeline. This introspection is invaluable for confirming that preprocessing ran correctly on the right columns.

# After fitting the preprocessor:
preprocessor.fit(X_train)

# Access the numeric sub-pipeline
num_pipeline = preprocessor.named_transformers_['num']
print('Imputer strategy:', num_pipeline['impute'].strategy)
print('Scaler mean:', num_pipeline['scale'].mean_)

# Access the OHE categories
cat_pipeline = preprocessor.named_transformers_['cat']
print('OHE categories:', cat_pipeline['ohe'].categories_)

Persisting and Reloading a Fitted ColumnTransformer

A fitted ColumnTransformer (or the full pipeline containing it) stores all learned parameters: scaling means and standard deviations, encoder category lists, imputer fill values. Serialise it with joblib.dump() and reload it later without re-training. This is how preprocessing is deployed to production: the fitted transformer ensures that incoming data is processed identically to training data, so predictions are consistent and correct across sessions.

import joblib

# Save the fitted pipeline
joblib.dump(pipeline, 'model_pipeline.joblib')

# Load and use in production without re-training
loaded_pipeline = joblib.load('model_pipeline.joblib')

# New raw data in the same format as training data
X_new = pd.DataFrame({
    'age': [30], 'income': [65000],
    'city': ['NYC'], 'education': ['master']
})

prediction = loaded_pipeline.predict(X_new)
print('Prediction:', prediction)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: how ColumnTransformer applies different transformations to different column subsets simultaneously, how to nest it inside a Pipeline for leak-free cross-validation, and how to compose imputation and encoding together using sub-pipelines. Next up we explore the K-Nearest Neighbors algorithm and how it classifies new data points using distance.

Häufig gestellte Fragen

Ist die Lektion „Schritte mit ColumnTransformer kombinieren“ kostenlos?

Ja — der vollständige Text von „Schritte mit ColumnTransformer kombinieren“ ist hier im Web kostenlos zu lesen. Um sie interaktiv zu üben (integrierter Code-Editor und 24/7 KI-Tutor) und den Rest des Machine Learning Academy-Kurses freizuschalten, upgrade auf CoddyKit PRO. Der Machine Learning Academy-Kurs umfasst insgesamt 4 Lektionen.

Was lerne ich in „Schritte mit ColumnTransformer kombinieren“?

Wenden Sie mit ColumnTransformer gleichzeitig unterschiedliche Vorverarbeitungsschritte auf verschiedene Spaltentypen an und erzeugen Sie eine einheitliche, bereinigte Feature-Matrix. Du übst Machine Learning Academy mit praktischem Code, den du direkt im Browser ausführst, und ein 24/7 KI-Tutor beantwortet deine Fragen während du die Lektion bearbeitest.

Brauche ich Erfahrung, um Machine Learning Academy zu starten?

Keine Vorkenntnisse erforderlich. Machine Learning Academy auf CoddyKit ist für Anfänger bis fortgeschrittene Lernende strukturiert, sodass du hier starten oder von Anfang an beginnen und in deinem eigenen Tempo voranschreiten kannst. Dies ist Lektion 4 von 4.

Wie lange dauert die Lektion „Schritte mit ColumnTransformer kombinieren“?

Die meisten CoddyKit-Lektionen dauern etwa 5–10 Minuten. Jede ist kompakt und interaktiv, sodass du stetig Fortschritte machst und genau dort weitermachst, wo du aufgehört hast – im Web und in der App.

Kann ich in dieser Machine Learning Academy-Lektion Code schreiben und ausführen?

Ja. Jede Machine Learning Academy-Lektion enthält einen integrierten Code-Editor, sodass du echten Code direkt in deinem Browser schreibst und ausführst und sofort KI-Feedback erhältst — ohne lokale Einrichtung erforderlich.

Alle Lektionen in diesem Kurs

  1. Fehlende Werte behandeln: Löschen, imputieren und markieren
  2. Feature-Skalierung: StandardScaler und MinMaxScaler
  3. Kategoriale Variablen codieren: OrdinalEncoder und OneHotEncoder
  4. Schritte mit ColumnTransformer kombinieren
← Zurück zu Machine Learning Academy