GroupBy-Transformation und -Filterung
Verwenden Sie transform(), um Gruppenstatistiken als Spalte hinzuzufügen, und filter(), um nur Gruppen zu behalten, die eine Bedingung erfüllen.
GroupBy-Transformation und -Filterung ist eine kostenlose Pandas & NumPy Academy-Lektion auf CoddyKit. Dies ist Lektion 4 von 4. Du kannst die komplette Lektion unten kostenlos lesen – dann übst du sie direkt im Browser mit einem integrierten Code-Editor und einem KI-Tutor rund um die Uhr. Sie ist Teil des Pandas & NumPy Academy-Lernpfads, und dein Fortschritt wird über Web und CoddyKit-App synchronisiert. Der Pandas & NumPy Academy-Kurs umfasst insgesamt 4 Lektionen.
Teile dieser Lektion wurden noch nicht übersetzt und werden auf Englisch angezeigt.
Beyond Aggregation
After mastering agg(), the natural question is: what if you want to keep all original rows but enrich them with group-level statistics? Or keep only the groups that meet a condition? This is where transform() and filter() come in. These two methods extend GroupBy beyond simple summarisation into feature engineering and data selection.
Understanding transform()
transform() applies a function to each group and returns a result with the same shape as the original DataFrame — one value per original row. The group result is broadcast back to each row that belongs to that group. This makes it ideal for adding group-level statistics as new columns without changing the row count.
import pandas as pd
df = pd.DataFrame({
'dept': ['Eng', 'HR', 'Eng', 'HR', 'Eng'],
'salary': [90000, 60000, 95000, 62000, 88000]
})
# Add a column with each employee's department average salary
df['dept_avg'] = df.groupby('dept')['salary'].transform('mean')
print(df)
# dept salary dept_avg
# 0 Eng 90000 91000.000
# 1 HR 60000 61000.000
# 2 Eng 95000 91000.000
# 3 HR 62000 61000.000
# 4 Eng 88000 91000.000Common transform() Use Cases
Common applications of transform() include: adding a group mean to normalise values, adding a group sum to compute per-row percentage of total, and adding a group rank to see how each member compares within its group. All of these preserve the original shape and index, making the result immediately usable alongside the original columns.
# Percentage of each employee's salary within their department total
df['pct_of_dept'] = (
df['salary'] / df.groupby('dept')['salary'].transform('sum') * 100
).round(1)
print(df[['dept', 'salary', 'pct_of_dept']])
# dept salary pct_of_dept
# Eng 90000 33.1
# HR 60000 49.2
# Eng 95000 34.9Using a Custom Function in transform()
Just like agg(), transform() accepts any callable in addition to string names. The function receives a Series of values for one group and must return a Series of the same length, or a scalar (which is then broadcast). Returning a scalar is the most common use case — returning a Series of a different length will raise an error.
# Z-score normalisation within each department
def zscore(s):
return (s - s.mean()) / s.std()
df['salary_zscore'] = df.groupby('dept')['salary'].transform(zscore)
print(df[['dept', 'salary', 'salary_zscore']].round(2))
# dept salary salary_zscore
# Eng 90000 -0.51
# HR 60000 -0.71
# Eng 95000 1.03agg() vs transform() Side by Side
The key difference: agg() reduces the number of rows (one per group), while transform() preserves the number of rows (one per original row). Use agg() to build a summary table. Use transform() to add group-level information as a new feature column in the original DataFrame.
g = df.groupby('dept')['salary']
# agg: 2 rows (one per unique dept)
print(g.agg('mean'))
# dept
# Eng 91000.0
# HR 61000.0
# transform: 5 rows (one per original row)
print(g.transform('mean'))
# 0 91000.0
# 1 61000.0
# 2 91000.0
# 3 61000.0
# 4 91000.0Understanding filter()
filter() keeps or discards entire groups based on a boolean function. You pass a function that receives a sub-DataFrame for one group and returns True (keep the group) or False (drop the group). The result is a subset of the original DataFrame containing only rows from the groups that passed the test.
df2 = pd.DataFrame({
'dept': ['Eng', 'HR', 'Eng', 'HR', 'Eng', 'Legal'],
'salary': [90000, 60000, 95000, 62000, 88000, 70000]
})
# Keep only departments with at least 2 employees
big_depts = df2.groupby('dept').filter(lambda g: len(g) >= 2)
print(big_depts)
# dept, salary rows: Eng(3) and HR(2) remain; Legal(1) droppedFilter by Group Aggregate Value
A very common use of filter() is keeping groups whose aggregate value meets a threshold. For example, keep only departments where the average salary exceeds a target, or keep only product categories with total sales above a minimum. This lets you remove low-volume groups before further analysis.
# Keep only departments where average salary > 80000
high_paying = df2.groupby('dept').filter(
lambda g: g['salary'].mean() > 80000
)
print(high_paying)
# dept salary
# 0 Eng 90000
# 2 Eng 95000
# 4 Eng 88000
# (HR avg is 61000, filtered out)Combining transform() and filter()
You can apply transform() and filter() sequentially to enrich your data and then narrow it down. First use filter() to remove irrelevant groups, then use transform() on the filtered result to add group-level features. The combination gives you a clean, feature-rich subset ready for modelling or reporting.
# Step 1: keep only large departments
filtered = df2.groupby('dept').filter(lambda g: len(g) >= 2)
# Step 2: add group mean salary to the filtered result
filtered = filtered.copy()
filtered['dept_avg'] = filtered.groupby('dept')['salary'].transform('mean')
print(filtered)transform() for Forward Filling Within Groups
transform() is also useful with non-numeric functions. A popular pattern is filling missing values within a group using the group's forward fill or median, which is much better than a global fill. Pass a lambda that calls fillna() on the group Series, and the result has the same index as the original DataFrame.
import numpy as np
df3 = pd.DataFrame({
'dept': ['Eng', 'Eng', 'HR', 'HR', 'Eng'],
'salary': [90000, np.nan, 60000, np.nan, 88000]
})
# Fill NaN with the group mean
df3['salary_filled'] = df3.groupby('dept')['salary'].transform(
lambda s: s.fillna(s.mean())
)
print(df3)Practical Pattern: Relative Standing
One powerful business use case is computing each row's standing relative to its group. By combining transform() with arithmetic, you can add columns like: salary minus group average (deviation), salary as a fraction of group total, or boolean flag for whether this employee earns above the group median. These features are extremely useful for dashboards and ML models.
df['dept_total'] = df.groupby('dept')['salary'].transform('sum')
df['pct_of_total'] = (df['salary'] / df['dept_total'] * 100).round(1)
df['above_avg'] = df['salary'] > df.groupby('dept')['salary'].transform('mean')
print(df[['dept', 'salary', 'pct_of_total', 'above_avg']])Performance Considerations
Both transform() and filter() with built-in string functions are fast because they use optimised code paths. However, when you pass a lambda or custom Python function, Pandas must call that function once per group, which can be slow on data with many groups. For maximum performance with large datasets, check whether your custom logic can be expressed using a built-in string function instead.
# Slower: custom lambda (called once per group)
df['dept_mean_slow'] = df.groupby('dept')['salary'].transform(lambda s: s.mean())
# Faster: built-in string shortcut (vectorised C path)
df['dept_mean_fast'] = df.groupby('dept')['salary'].transform('mean')
# Both give identical results, but the built-in is significantly fasterQuick Check
Test your understanding of GroupBy transform() and filter() from this lesson.
Lesson Recap
In this lesson you learned: transform() returns group statistics broadcast back to the original row count, making it ideal for adding group-level features; filter() keeps or removes entire groups based on a boolean condition; and combining both lets you build rich, filtered datasets for downstream analysis. Next up we explore how to combine DataFrames using pd.concat.
Häufig gestellte Fragen
Ist die Lektion „GroupBy-Transformation und -Filterung“ kostenlos?
Ja — der vollständige Text von „GroupBy-Transformation und -Filterung“ ist hier im Web kostenlos zu lesen. Um sie interaktiv zu üben (integrierter Code-Editor und 24/7 KI-Tutor) und den Rest des Pandas & NumPy Academy-Kurses freizuschalten, upgrade auf CoddyKit PRO. Der Pandas & NumPy Academy-Kurs umfasst insgesamt 4 Lektionen.
Was lerne ich in „GroupBy-Transformation und -Filterung“?
Verwenden Sie transform(), um Gruppenstatistiken als Spalte hinzuzufügen, und filter(), um nur Gruppen zu behalten, die eine Bedingung erfüllen. Du übst Pandas & NumPy Academy mit praktischem Code, den du direkt im Browser ausführst, und ein 24/7 KI-Tutor beantwortet deine Fragen während du die Lektion bearbeitest.
Brauche ich Erfahrung, um Pandas & NumPy Academy zu starten?
Keine Vorkenntnisse erforderlich. Pandas & NumPy Academy auf CoddyKit ist für Anfänger bis fortgeschrittene Lernende strukturiert, sodass du hier starten oder von Anfang an beginnen und in deinem eigenen Tempo voranschreiten kannst. Dies ist Lektion 4 von 4.
Wie lange dauert die Lektion „GroupBy-Transformation und -Filterung“?
Die meisten CoddyKit-Lektionen dauern etwa 5–10 Minuten. Jede ist kompakt und interaktiv, sodass du stetig Fortschritte machst und genau dort weitermachst, wo du aufgehört hast – im Web und in der App.
Kann ich in dieser Pandas & NumPy Academy-Lektion Code schreiben und ausführen?
Ja. Jede Pandas & NumPy Academy-Lektion enthält einen integrierten Code-Editor, sodass du echten Code direkt in deinem Browser schreibst und ausführst und sofort KI-Feedback erhältst — ohne lokale Einrichtung erforderlich.
Alle Lektionen in diesem Kurs
- Das Split-Apply-Combine-Muster
- GroupBy mit einem und mehreren Schlüsseln
- Die Methode agg()
- GroupBy-Transformation und -Filterung