Pandas & NumPy Academy · Lektion

iterrows und Python-Schleifen vermeiden

Ersetzen Sie zeilenweise Schleifen durch vektorisierte Spaltenoperationen, np.where und pd.cut, um eine Beschleunigung um das 10- bis 100-Fache zu erreichen.

iterrows und Python-Schleifen vermeiden ist eine kostenlose Pandas & NumPy Academy-Lektion auf CoddyKit. Dies ist Lektion 2 von 4. Du kannst die komplette Lektion unten kostenlos lesen – dann übst du sie direkt im Browser mit einem integrierten Code-Editor und einem KI-Tutor rund um die Uhr. Sie ist Teil des Pandas & NumPy Academy-Lernpfads, und dein Fortschritt wird über Web und CoddyKit-App synchronisiert. Der Pandas & NumPy Academy-Kurs umfasst insgesamt 4 Lektionen.

Teile dieser Lektion wurden noch nicht übersetzt und werden auf Englisch angezeigt.

The Cost of Row-by-Row Iteration

Pandas is built on NumPy, which processes entire arrays at once using compiled C code. When you iterate row by row with for _, row in df.iterrows(), you bypass this and fall back to pure Python — each row is extracted as a Series object, and the loop runs in the Python interpreter at roughly 100-1000x the cost of an equivalent vectorised operation. For a 1-million-row DataFrame, this can mean minutes instead of milliseconds.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({'a': np.random.randn(10000), 'b': np.random.randn(10000)})

# Slow: iterrows loop
def loop_version(df):
    result = []
    for _, row in df.iterrows():
        result.append(row['a'] + row['b'])
    return pd.Series(result)

# Fast: vectorised
t_loop = timeit.timeit(lambda: loop_version(df), number=5)
t_vec = timeit.timeit(lambda: df['a'] + df['b'], number=500)

print(f'Loop (5 runs):       {t_loop:.3f}s total')
print(f'Vectorised (500 runs): {t_vec:.3f}s total')
print(f'Speedup: ~{(t_loop/5)/(t_vec/500):.0f}x')

Replace Conditional Loops with np.where

np.where(condition, value_if_true, value_if_false) is the vectorised equivalent of an element-wise if/else. Instead of iterating rows and writing an if statement, pass the condition and both outcome values as arrays. np.where computes this in C for the entire column at once, making it 50-200x faster than an equivalent Python loop for large DataFrames.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({'price': np.random.uniform(10, 100, 100000)})

# Slow: iterrows
def label_loop(df):
    labels = []
    for _, row in df.iterrows():
        if row['price'] > 50:
            labels.append('expensive')
        else:
            labels.append('cheap')
    return labels

# Fast: np.where
def label_vectorised(df):
    return np.where(df['price'] > 50, 'expensive', 'cheap')

# Verify equivalence on a small subset
assert list(label_loop(df.head(100))) == list(label_vectorised(df.head(100)))
print('Results match!')
print('Fast version result sample:', label_vectorised(df)[:5])

Multiple Conditions with np.select

For three or more conditions, use np.select(condlist, choicelist, default=) instead of nested np.where. Pass a list of boolean arrays and a list of corresponding output values. The first matching condition determines the output; unmatched rows get the default value. This replaces complex if/elif/else chains inside loops with a clean, vectorised expression.

import pandas as pd
import numpy as np

np.random.seed(0)
df = pd.DataFrame({'score': np.random.randint(0, 101, 20)})

conditions = [
    df['score'] >= 90,
    df['score'] >= 75,
    df['score'] >= 60
]
choices = ['A', 'B', 'C']

df['grade'] = np.select(conditions, choices, default='F')
print(df.sort_values('score', ascending=False).head(10))

Vectorised String Operations via .str

String manipulation is a common source of slow loops. The Pandas .str accessor provides vectorised equivalents of all Python string methods: .str.lower(), .str.strip(), .str.replace(), .str.contains(), and many more. Using .str methods is 10-50x faster than iterating rows and calling Python string methods manually.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
names = ['Alice Smith', 'BOB JONES', '  carol  ', 'Dave Brown'] * 25000
df = pd.DataFrame({'name': names})

# Slow: loop
def clean_loop(df):
    return [n.strip().title() for n in df['name']]

# Fast: .str accessor
def clean_vectorised(df):
    return df['name'].str.strip().str.title()

t1 = timeit.timeit(lambda: clean_loop(df), number=10)
t2 = timeit.timeit(lambda: clean_vectorised(df), number=50)

print(f'Loop (10 runs): {t1:.3f}s')
print(f'.str (50 runs): {t2:.3f}s')
print(f'Speedup: {(t1/10)/(t2/50):.0f}x')
print('Sample:', clean_vectorised(df).head(4).tolist())

Using pd.cut and pd.qcut Instead of Loops

pd.cut() and pd.qcut() vectorise binning operations that are often written as loops with multiple if/elif conditions. If you find yourself writing 'if value < 18: age_group = child elif value < 65: age_group = adult' inside a row loop, replace it with a single pd.cut() call that processes the entire column at once in C-speed.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({'age': np.random.randint(0, 90, 100000)})

# Slow: loop with conditionals
def categorise_loop(df):
    result = []
    for age in df['age']:
        if age < 18:
            result.append('child')
        elif age < 65:
            result.append('adult')
        else:
            result.append('senior')
    return result

# Fast: pd.cut
def categorise_cut(df):
    return pd.cut(df['age'], bins=[0, 18, 65, 100],
                  labels=['child', 'adult', 'senior'],
                  right=False)

assert list(categorise_loop(df.head(5))) == list(categorise_cut(df.head(5)).astype(str))
print('Fast version sample:', categorise_cut(df).head(5).tolist())

apply() is Not Always the Answer

Many tutorials recommend replacing loops with .apply(func), but apply is still a Python-level loop internally — it is only marginally faster than iterrows and much slower than true vectorisation. Reserve apply for cases where no vectorised alternative exists (complex multi-column logic). If a built-in Pandas or NumPy function can express the operation, always prefer it over apply.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({'x': np.random.randn(100000)})

# These all do the same thing — compare performance
t1 = timeit.timeit(lambda: df['x'].apply(lambda v: v**2), number=20)
t2 = timeit.timeit(lambda: df['x'] ** 2, number=200)
t3 = timeit.timeit(lambda: np.square(df['x']), number=200)

print(f'apply(v**2): {t1/20*1000:.2f} ms per run')
print(f'** operator: {t2/200*1000:.2f} ms per run')
print(f'np.square(): {t3/200*1000:.2f} ms per run')

Replacing groupby Loops with agg and transform

A common pattern is iterating over groups manually: for name, group in df.groupby('cat'): do_something(group). This is slow for the same reasons iterrows is slow. Replace it with groupby().agg() for producing summary statistics, or groupby().transform() for broadcasting group-level statistics back to the original row positions.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({
    'cat': np.random.choice(['A','B','C','D'], 100000),
    'val': np.random.randn(100000)
})

# Slow: manual loop to add group mean
def slow_group_mean(df):
    means = {}
    for name, group in df.groupby('cat'):
        means[name] = group['val'].mean()
    return df['cat'].map(means)

# Fast: transform
def fast_group_mean(df):
    return df.groupby('cat')['val'].transform('mean')

t1 = timeit.timeit(lambda: slow_group_mean(df), number=10)
t2 = timeit.timeit(lambda: fast_group_mean(df), number=100)
print(f'Loop:       {t1/10*1000:.1f} ms')
print(f'transform:  {t2/100*1000:.1f} ms')
print(f'Speedup: {(t1/10)/(t2/100):.0f}x')

When iterrows Is Acceptable

Despite being slow, there are legitimate uses for iterrows and itertuples: printing or logging information from rows (performance is irrelevant), constructing API payloads where each row drives an HTTP call (the network latency dominates), and debugging specific rows with complex conditions. If you have fewer than 1,000 rows and the operation runs once, the difference between a loop and vectorisation is milliseconds — not worth the code complexity.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'order_id': [101, 102, 103],
    'product': ['Widget', 'Gadget', 'Doohickey'],
    'amount': [29.99, 49.99, 9.99]
})

# Acceptable use: building a structured log (small data, one-time)
for _, row in df.iterrows():
    print(f'Order {row["order_id"]}: {row["product"]} — ${row["amount"]:.2f}')

itertuples Is Faster Than iterrows

If you must iterate rows in Python (because no vectorised equivalent exists), use itertuples() instead of iterrows(). It returns each row as a named tuple rather than a Pandas Series, avoiding the Series construction overhead. This makes it typically 5-10x faster than iterrows. Access values with attribute notation: row.column_name. Avoid if column names have spaces (not valid attribute names).

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({'a': np.random.randn(10000), 'b': np.random.randn(10000)})

t1 = timeit.timeit(
    lambda: [row.a + row.b for row in df.itertuples()], number=10)
t2 = timeit.timeit(
    lambda: [row['a'] + row['b'] for _, row in df.iterrows()], number=10)

print(f'itertuples: {t1/10*1000:.1f} ms')
print(f'iterrows:   {t2/10*1000:.1f} ms')
print(f'itertuples speedup: {t2/t1:.1f}x')

Vectorisation Pattern Checklist

Before writing a loop, ask these questions:

Is the operation element-wise on one column? → Use a column arithmetic expression or NumPy ufunc.
Does it involve conditional logic? → Use np.where (2 conditions) or np.select (3+).
Does it bin values into ranges? → Use pd.cut or pd.qcut.
Does it apply string operations? → Use the .str accessor.
Does it aggregate within groups? → Use groupby().agg() or groupby().transform().

Only fall back to apply() or itertuples() if none of the above applies.

Real Speedup Example: Price Calculation

Here is a complete before/after refactoring of a common business calculation — applying tiered discounts based on order quantity. The loop version is readable but unacceptably slow for large DataFrames. The vectorised version using np.select achieves the same result in one tenth of the time.

import pandas as pd
import numpy as np
import timeit

np.random.seed(0)
df = pd.DataFrame({
    'price': np.random.uniform(10, 200, 100000),
    'qty': np.random.randint(1, 500, 100000)
})

# BEFORE: loop with conditionals
def slow_discount(df):
    result = []
    for _, row in df.iterrows():
        if row['qty'] >= 100:
            disc = 0.2
        elif row['qty'] >= 50:
            disc = 0.1
        elif row['qty'] >= 10:
            disc = 0.05
        else:
            disc = 0.0
        result.append(row['price'] * (1 - disc))
    return pd.Series(result)

# AFTER: np.select
def fast_discount(df):
    conditions = [df['qty'] >= 100, df['qty'] >= 50, df['qty'] >= 10]
    discounts = [0.20, 0.10, 0.05]
    disc = np.select(conditions, discounts, default=0.0)
    return df['price'] * (1 - disc)

assert slow_discount(df.head(200)).round(4).equals(fast_discount(df.head(200)).reset_index(drop=True).round(4))
print('Both versions agree!')

Quick Check

Test your understanding of vectorisation from this lesson.

Lesson Recap

In this lesson you learned: iterrows is 100-1000x slower than vectorised operations, np.where and np.select replace conditional loops, the .str accessor vectorises string operations, and groupby().transform() replaces manual group iteration. When you must iterate, use itertuples over iterrows for a 5-10x improvement. Next up we explore efficient data types — downcasting numerics and using Categorical to reduce memory by up to 70%.

Häufig gestellte Fragen

Ist die Lektion „iterrows und Python-Schleifen vermeiden“ kostenlos?

Ja — der vollständige Text von „iterrows und Python-Schleifen vermeiden“ ist hier im Web kostenlos zu lesen. Um sie interaktiv zu üben (integrierter Code-Editor und 24/7 KI-Tutor) und den Rest des Pandas & NumPy Academy-Kurses freizuschalten, upgrade auf CoddyKit PRO. Der Pandas & NumPy Academy-Kurs umfasst insgesamt 4 Lektionen.

Was lerne ich in „iterrows und Python-Schleifen vermeiden“?

Ersetzen Sie zeilenweise Schleifen durch vektorisierte Spaltenoperationen, np.where und pd.cut, um eine Beschleunigung um das 10- bis 100-Fache zu erreichen. Du übst Pandas & NumPy Academy mit praktischem Code, den du direkt im Browser ausführst, und ein 24/7 KI-Tutor beantwortet deine Fragen während du die Lektion bearbeitest.

Brauche ich Erfahrung, um Pandas & NumPy Academy zu starten?

Keine Vorkenntnisse erforderlich. Pandas & NumPy Academy auf CoddyKit ist für Anfänger bis fortgeschrittene Lernende strukturiert, sodass du hier starten oder von Anfang an beginnen und in deinem eigenen Tempo voranschreiten kannst. Dies ist Lektion 2 von 4.

Wie lange dauert die Lektion „iterrows und Python-Schleifen vermeiden“?

Die meisten CoddyKit-Lektionen dauern etwa 5–10 Minuten. Jede ist kompakt und interaktiv, sodass du stetig Fortschritte machst und genau dort weitermachst, wo du aufgehört hast – im Web und in der App.

Kann ich in dieser Pandas & NumPy Academy-Lektion Code schreiben und ausführen?

Ja. Jede Pandas & NumPy Academy-Lektion enthält einen integrierten Code-Editor, sodass du echten Code direkt in deinem Browser schreibst und ausführst und sofort KI-Feedback erhältst — ohne lokale Einrichtung erforderlich.

Alle Lektionen in diesem Kurs

Profiling mit timeit und memory_profiler
iterrows und Python-Schleifen vermeiden
Effiziente Datentypen zur Speicherreduzierung
Chunkweises Lesen großer Dateien

← Zurück zu Pandas & NumPy Academy