排序索引的性能优势
使用 sort_index() 对 MultiIndex 排序，使用 timeit 测量切片性能，并用 is_monotonic_increasing 进行保护性检查。
排序索引的性能优势 是 CoddyKit 上的免费 Pandas & NumPy Academy 课时。 这是第 4 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Pandas & NumPy Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Pandas & NumPy Academy 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
Why Index Sorting Matters for Performance
A sorted index enables Pandas to use binary search (O(log n)) instead of a full linear scan (O(n)) when looking up label ranges. For a DataFrame with one million rows, binary search finds the target range in about 20 comparisons vs. up to one million comparisons with a linear scan. This makes slice operations on sorted MultiIndexes orders of magnitude faster than on unsorted ones — and the difference becomes critical in production pipelines processing millions of rows.
import pandas as pd
import numpy as np
np.random.seed(42)
# Create a large DataFrame with a MultiIndex
countries = ['DE', 'UK', 'USA', 'FR', 'JP']
dates = pd.date_range('2020-01-01', periods=200)
mi = pd.MultiIndex.from_product([countries, dates], names=['country', 'date'])
df = pd.DataFrame({'value': np.random.randn(len(mi))}, index=mi)
print(f'DataFrame shape: {df.shape}')
print(f'Index is sorted: {df.index.is_monotonic_increasing}')Checking if an Index Is Sorted
Use df.index.is_monotonic_increasing to check whether the index is sorted in ascending order. This returns a boolean. For a MultiIndex, Pandas checks sorting lexicographically across all levels. Always check this before performing slice operations with .loc[start:end] on a MultiIndex — an unsorted index will either raise UnsortedIndexError or silently return incorrect results depending on the Pandas version.
import pandas as pd
# Sorted MultiIndex
tuples_sorted = [('A', 1), ('A', 2), ('B', 1), ('B', 2)]
mi_sorted = pd.MultiIndex.from_tuples(tuples_sorted)
df_sorted = pd.DataFrame({'v': [10, 20, 30, 40]}, index=mi_sorted)
# Unsorted MultiIndex
tuples_unsorted = [('B', 2), ('A', 1), ('B', 1), ('A', 2)]
mi_unsorted = pd.MultiIndex.from_tuples(tuples_unsorted)
df_unsorted = pd.DataFrame({'v': [10, 20, 30, 40]}, index=mi_unsorted)
print('Sorted index is_monotonic_increasing:', df_sorted.index.is_monotonic_increasing)
print('Unsorted index is_monotonic_increasing:', df_unsorted.index.is_monotonic_increasing)Sorting with sort_index()
df.sort_index() returns a new DataFrame with rows sorted by index label in ascending order. Use ascending=False for descending. For a MultiIndex, sorting is lexicographic: it sorts by the outermost level first, then by inner levels within each outer group. Always sort after any operation that might disorder the index — such as pd.concat, filtering, or appending new rows.
import pandas as pd
import numpy as np
np.random.seed(0)
countries = ['USA', 'UK', 'DE']
years = [2021, 2022, 2023]
mi = pd.MultiIndex.from_product([countries, years], names=['country', 'year'])
df = pd.DataFrame({'gdp': np.random.randint(3000, 26000, 9)}, index=mi)
print('Before sort_index():')
print(df.head(4))
df_sorted = df.sort_index()
print('\nAfter sort_index():')
print(df_sorted.head(4))
print('Is sorted:', df_sorted.index.is_monotonic_increasing)Measuring Lookup Time with timeit
Python's timeit module measures how long a statement takes to execute by running it many times and averaging. Use it to benchmark sorted vs. unsorted index lookups. In IPython/Jupyter, the %timeit magic provides the same functionality with nicer output. Benchmarking is the only reliable way to confirm that a performance optimisation actually helped — never assume a change is faster without measuring it.
import pandas as pd
import numpy as np
import timeit
np.random.seed(0)
N = 500000
idx = np.random.choice(['A','B','C','D','E'], N)
df_unsorted = pd.DataFrame({'v': np.random.randn(N)}, index=idx)
df_sorted = df_unsorted.sort_index()
# Time label lookup: sorted vs unsorted
t_unsorted = timeit.timeit(lambda: df_unsorted.loc['C'], number=100)
t_sorted = timeit.timeit(lambda: df_sorted.loc['C'], number=100)
print(f'Unsorted lookup (100 runs): {t_unsorted:.3f}s')
print(f'Sorted lookup (100 runs): {t_sorted:.3f}s')
print(f'Speedup: {t_unsorted/t_sorted:.1f}x')PerformanceWarning from Unsorted MultiIndex
Pandas emits a PerformanceWarning when you slice a MultiIndex that is not lexicographically sorted: 'indexing past lexsort depth may impact performance'. This warning means Pandas had to fall back to a linear scan instead of binary search. While it still returns correct results in simple cases, it can return incorrect results when slicing inner levels of an unsorted multi-level index. Treat this warning as an error and fix the root cause by sorting the index.
import pandas as pd
import warnings
# Create an unsorted MultiIndex and trigger the warning
tuples = [('B', 2), ('A', 1), ('B', 1), ('A', 2)]
mi = pd.MultiIndex.from_tuples(tuples, names=['letter', 'num'])
df = pd.DataFrame({'v': [10, 20, 30, 40]}, index=mi)
print('Index sorted?', df.index.is_monotonic_increasing)
# This may trigger PerformanceWarning in some Pandas versions
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter('always')
try:
result = df.loc['A':'B', :]
print('Result:', result)
if w:
print('Warning:', str(w[0].message))
except Exception as e:
print('Error (common with newer Pandas):', type(e).__name__)Benchmarking Sorted vs Unsorted MultiIndex Slicing
Sorted MultiIndex slicing is dramatically faster because Pandas can binary-search both the outer and inner level arrays. Let's benchmark slicing a large MultiIndex with 1 million rows — a common size in production analytics pipelines. The sorted version avoids the linear scan and consistently shows 5-50× speedups depending on the selectivity of the slice.
import pandas as pd
import numpy as np
import timeit
np.random.seed(42)
countries = ['DE', 'UK', 'USA', 'FR', 'JP']
dates = pd.date_range('2010-01-01', periods=200000)
# Sample a random subset for timing test
sample_countries = np.random.choice(countries, 100000)
sample_dates = np.random.choice(dates, 100000)
df = pd.DataFrame({
'country': sample_countries,
'date': sample_dates,
'value': np.random.randn(100000)
}).set_index(['country', 'date'])
df_sorted = df.sort_index()
print('Dataset size:', len(df))
print('Sorted:', df_sorted.index.is_monotonic_increasing)
t = timeit.timeit(lambda: df_sorted.loc['USA'], number=50)
print(f'Sorted lookup (50 runs): {t:.3f}s')sort_index with level Parameter
For a MultiIndex, you can sort by a specific level rather than all levels using the level parameter: df.sort_index(level='year'). This is useful when you want to preserve the outer-level grouping but reorder rows within each outer group. The sort_remaining=True argument (default) also sorts any unsorted levels beyond the specified one, ensuring full lexicographic ordering.
import pandas as pd
import numpy as np
countries = ['USA', 'UK']
years = [2023, 2021, 2022] # deliberately unordered
mi = pd.MultiIndex.from_product([countries, years], names=['country', 'year'])
df = pd.DataFrame({'v': range(6)}, index=mi)
print('Before sorting by year level:')
print(df)
# Sort by the inner level (year) only
df_ysorted = df.sort_index(level='year')
print('\nAfter sort_index(level="year"):')
print(df_ysorted)is_monotonic_increasing as a Pipeline Guard
In production pipelines, add a sorting guard at the start of any function that receives a DataFrame with a MultiIndex and performs slicing. If the index is not sorted, sort it automatically and log a warning. This prevents silent performance degradation or incorrect results when upstream code changes the DataFrame order. A guard at the function boundary is more reliable than assuming callers always pass sorted data.
import pandas as pd
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def safe_slice(df, key):
'''Slice a MultiIndex DataFrame, sorting if necessary.'''
if not df.index.is_monotonic_increasing:
logger.warning('Index not sorted — sorting now. This is a performance cost.')
df = df.sort_index()
return df.loc[key]
# Test with an unsorted DataFrame
tuples = [('B', 2), ('A', 1), ('B', 1), ('A', 2)]
mi = pd.MultiIndex.from_tuples(tuples, names=['letter', 'num'])
df = pd.DataFrame({'v': [10, 20, 30, 40]}, index=mi)
result = safe_slice(df, 'A')
print('Slice result for A:')
print(result)Sorted Index for Binary Search on Simple Indices
The performance benefits of sorting apply to regular (non-multi) indices too. A DatetimeIndex used in time series analysis performs much faster date range slicing when sorted. A string index sorted alphabetically allows binary search for label lookups. For a large Series of stock prices indexed by timestamp, sorting the DatetimeIndex can turn a 100ms slice into a sub-millisecond operation.
import pandas as pd
import numpy as np
import timeit
np.random.seed(0)
# Random timestamps — unsorted
timestamps = pd.date_range('2020-01-01', periods=500000, freq='min')
shuffled = np.random.permutation(timestamps)
prices = pd.Series(np.random.randn(500000), index=shuffled)
prices_sorted = prices.sort_index()
# Time a date range slice
t_unsorted = timeit.timeit(lambda: prices['2020-06-01':'2020-06-30'], number=20)
t_sorted = timeit.timeit(lambda: prices_sorted['2020-06-01':'2020-06-30'], number=20)
print(f'Unsorted: {t_unsorted:.3f}s')
print(f'Sorted: {t_sorted:.3f}s')
print(f'Speedup: {t_unsorted/t_sorted:.0f}x')Memory Cost of Sorting
Sorting is not free — sort_index() creates a new copy of the DataFrame (unless using inplace=True which modifies in place). For very large DataFrames, this doubles peak memory usage temporarily. A pragmatic strategy is to sort once at load time and keep the sorted version throughout the pipeline, rather than sorting repeatedly. If memory is tight, sort in place with df.sort_index(inplace=True) to avoid the temporary copy.
import pandas as pd
import numpy as np
np.random.seed(0)
countries = ['DE', 'UK', 'USA']
years = [2021, 2022, 2023]
mi = pd.MultiIndex.from_product([countries, years], names=['country', 'year'])
df = pd.DataFrame({'v': np.random.randn(9)}, index=mi)
# Sort once at load time — best practice
df.sort_index(inplace=True) # no temporary copy
assert df.index.is_monotonic_increasing, 'Index must be sorted!'
print('Pipeline-ready DataFrame (sorted in place):')
print(df)Summary of Sorted Index Best Practices
Key rules for index performance: 1) Always call sort_index() after any operation that may disorder the index (concat, merge, append, filter). 2) Use is_monotonic_increasing as a guard in functions that slice the index. 3) Sort at load time and keep the sorted DataFrame throughout the pipeline to avoid repeated sorting. 4) For MultiIndex DataFrames, ensure all levels are sorted, not just the outermost. 5) Use timeit to verify that sorting actually provides the expected speedup in your specific pipeline.
Quick Check
Test your understanding of sorted index performance from this lesson.
Lesson Recap
In this lesson you learned: is_monotonic_increasing checks whether an index is sorted and binary search is available, sort_index() sorts in place or returns a sorted copy, and timeit measures the actual speedup to confirm the benefit. Next up we explore window functions — rolling and expanding statistics for time series and financial data.
常见问题解答
「排序索引的性能优势」课时是免费的吗？
是的 — 「排序索引的性能优势」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Pandas & NumPy Academy 课程的其余内容，请升级到 CoddyKit PRO。 Pandas & NumPy Academy 课程共包含 4 节课。
「排序索引的性能优势」这节课中我会学到什么？
使用 sort_index() 对 MultiIndex 排序，使用 timeit 测量切片性能，并用 is_monotonic_increasing 进行保护性检查。 你通过在浏览器中直接运行的动手代码来练习 Pandas & NumPy Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Pandas & NumPy Academy 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Pandas & NumPy Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 4 节课，共 4 节。
「排序索引的性能优势」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Pandas & NumPy Academy 课中编写并运行代码吗？
能。每节 Pandas & NumPy Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。