0Pricing
Pandas & NumPy Academy · 课时

分类数据类型

将低基数的字符串列转换为 pandas Categorical，以减少内存使用并加快分组操作。

分类数据类型 是 CoddyKit 上的免费 Pandas & NumPy Academy 课时。 这是第 3 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Pandas & NumPy Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Pandas & NumPy Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

What Is the Categorical Dtype?

The Categorical dtype in Pandas is designed for columns that contain a limited set of discrete values (low cardinality), such as gender, status, region, or product category. Instead of storing the full string for every row, Pandas stores the unique values (called categories) once and uses an integer code per row to reference them. This is similar to how databases use lookup tables or enum types.

import pandas as pd

df = pd.DataFrame({
    'region': ['North', 'South', 'East', 'North', 'East', 'South'] * 1000
})

# Without Categorical: object dtype stores every string
print('object dtype memory:', df['region'].memory_usage(deep=True))

# With Categorical: only stores 3 unique values + integer codes
df['region_cat'] = df['region'].astype('category')
print('category dtype memory:', df['region_cat'].memory_usage(deep=True))

Creating a Categorical Series

Convert a column to Categorical by calling .astype('category') on any Series. The resulting Categorical Series stores the unique values as .cat.categories and the integer position codes as .cat.codes. You can also create a Categorical directly with pd.Categorical() to specify the categories and their order up front.

import pandas as pd

s = pd.Series(['low', 'high', 'med', 'high', 'low'])
s_cat = s.astype('category')

print(s_cat.cat.categories)  # Index(['high', 'low', 'med'], dtype='object')
print(s_cat.cat.codes)       # integer codes per row
# 0    1
# 1    0
# 2    2
# 3    0
# 4    1

Memory Savings with Categorical

The memory savings from Categorical dtype depend on the cardinality ratio (unique values ÷ total rows). With 5 unique strings in a 1,000,000-row column, the object dtype stores 1 million string pointers (~50+ bytes each) while Categorical stores just 5 strings plus 1 million 8-bit integers. The savings can be 5-20x, turning a 50 MB column into 2-5 MB.

import pandas as pd
import numpy as np

n = 1_000_000
statuses = np.random.choice(['active', 'inactive', 'pending'], size=n)
df = pd.DataFrame({'status': statuses})

object_mem = df['status'].memory_usage(deep=True) / 1e6
df['status'] = df['status'].astype('category')
cat_mem = df['status'].memory_usage(deep=True) / 1e6

print(f'Object dtype:    {object_mem:.1f} MB')
print(f'Category dtype:  {cat_mem:.1f} MB')
print(f'Reduction:       {object_mem/cat_mem:.1f}x')

Ordered Categorical for Ranking

Sometimes categories have a natural order — e.g., 'low' < 'medium' < 'high', or T-shirt sizes XS < S < M < L < XL. An ordered Categorical captures this ranking, enabling meaningful comparison operators (<, >=, etc.) and ensuring groupby results are sorted in the correct semantic order rather than alphabetically.

import pandas as pd

df = pd.DataFrame({
    'priority': ['high', 'low', 'medium', 'high', 'low']
})

priority_type = pd.CategoricalDtype(
    categories=['low', 'medium', 'high'],
    ordered=True
)
df['priority'] = df['priority'].astype(priority_type)

# Comparison now works correctly
print(df['priority'] >= 'medium')
# 0     True
# 1    False
# 2     True
# 3     True
# 4    False

Sorting Ordered Categoricals

When you sort an ordered Categorical column, Pandas uses the defined category order rather than alphabetical order. This means 'low' comes before 'medium' before 'high' regardless of how they would sort as strings. This is critical for producing correctly ordered summary tables and charts.

import pandas as pd

df = pd.DataFrame({
    'severity': pd.Categorical(
        ['high', 'low', 'medium', 'low', 'high'],
        categories=['low', 'medium', 'high'],
        ordered=True
    ),
    'count': [5, 20, 8, 15, 3]
})

# sort_values respects the categorical order
print(df.sort_values('severity')[['severity', 'count']])
#   severity  count
# 1      low     20
# 3      low     15
# 2   medium      8
# 0     high      5
# 4     high      3

GroupBy Speed with Categorical

Pandas can optimise groupby() operations on Categorical columns because it knows all possible groups in advance. This can make groupby 2-5x faster on large DataFrames. Additionally, groupby on Categorical returns all categories — including empty ones — which ensures your summary tables always have a row for every expected group, even if some have zero observations.

import pandas as pd
import numpy as np

np.random.seed(0)
df = pd.DataFrame({
    'region': pd.Categorical(
        np.random.choice(['North', 'South', 'East', 'West'], 10),
        categories=['North', 'South', 'East', 'West']
    ),
    'sales': np.random.randint(100, 1000, 10)
})

# observed=False shows ALL categories even empty ones
print(df.groupby('region', observed=False)['sales'].sum())

Adding and Removing Categories

Use the .cat accessor to manage the category list. .cat.add_categories() adds new allowed values (useful before inserting new data), and .cat.remove_unused_categories() drops category labels that have no corresponding rows — handy after filtering. You cannot assign a value that is not in the categories without adding it first.

import pandas as pd

s = pd.Categorical(['a', 'b', 'a'], categories=['a', 'b', 'c'])
s = pd.Series(s)

print(s.cat.categories)  # Index(['a', 'b', 'c'], dtype='object')
print(s.value_counts())  # a:2, b:1, c:0

# Remove unused category 'c'
s = s.cat.remove_unused_categories()
print(s.cat.categories)  # Index(['a', 'b'], dtype='object')

Renaming Category Labels

.cat.rename_categories() lets you relabel categories without changing the underlying codes. This is useful when you want to display friendlier names (e.g., 'M' → 'Male') or fix inconsistent label capitalisation without converting back to object and remapping. The method accepts a list (positional) or a dictionary (targeted).

import pandas as pd

s = pd.Series(pd.Categorical(['M', 'F', 'M', 'F'], categories=['M', 'F']))

# Rename using a dict
s = s.cat.rename_categories({'M': 'Male', 'F': 'Female'})
print(s)
# 0      Male
# 1    Female
# 2      Male
# 3    Female
print(s.cat.categories)  # Index(['Male', 'Female'], dtype='object')

When NOT to Use Categorical

Categorical dtype is a poor choice when cardinality is high — if almost every row has a unique value (like user IDs, free-text comments, or UUIDs), the category index is nearly as large as the original data, giving no memory saving. A rule of thumb: use Categorical only when the number of unique values is less than roughly 50% of the total rows, and especially when unique values number in the tens or hundreds.

import pandas as pd
import numpy as np

df = pd.DataFrame({'user_id': range(1_000_000)})

# High-cardinality: every value is unique — don't use Categorical
object_mem = df['user_id'].astype(str).memory_usage(deep=True) / 1e6
cat_mem = df['user_id'].astype(str).astype('category').memory_usage(deep=True) / 1e6

print(f'String: {object_mem:.1f} MB')
print(f'Category: {cat_mem:.1f} MB')
# Category is WORSE for high-cardinality data!

Categorical in Pivot Tables and Groupby

Using Categorical ensures consistent output shape in groupby and pivot table results. Without Categorical, groups that happen to have zero observations are silently omitted from the result — which can cause misaligned output when comparing across different data slices. With Categorical and observed=False, all groups always appear in the output.

import pandas as pd

df = pd.DataFrame({
    'quarter': pd.Categorical(
        ['Q1', 'Q1', 'Q3'],
        categories=['Q1', 'Q2', 'Q3', 'Q4']
    ),
    'revenue': [100, 200, 300]
})

# observed=False includes Q2 and Q4 even though they have no rows
result = df.groupby('quarter', observed=False)['revenue'].sum()
print(result)
# quarter
# Q1    300
# Q2      0
# Q3    300
# Q4      0

Categorical and Machine Learning

Many scikit-learn estimators expect numeric input. To convert a Categorical column to numbers for a model, use .cat.codes (label encoding) or pd.get_dummies() (one-hot encoding). One-hot encoding avoids implying an ordinal relationship between categories. For ordered categoricals like priority levels, label encoding (the codes directly) is appropriate and carries the ordering information.

import pandas as pd

df = pd.DataFrame({
    'colour': pd.Categorical(['red', 'blue', 'green', 'red'])
})

# One-hot encode for unordered categories
one_hot = pd.get_dummies(df['colour'], prefix='colour')
print(one_hot)
#    colour_blue  colour_green  colour_red
# 0            0             0           1
# 1            1             0           0
# 2            0             1           0
# 3            0             0           1

Quick Check

Test your understanding of the Categorical dtype.

Lesson Recap

In this lesson you learned: Categorical dtype stores unique values once and uses integer codes per row, providing large memory savings for low-cardinality columns, ordered Categorical supports meaningful comparisons and correct sorting, and groupby with observed=False includes all categories even empty ones. Avoid Categorical for high-cardinality columns. Next up we parse date strings correctly with pd.to_datetime().

常见问题解答

「分类数据类型」课时是免费的吗？

是的 — 「分类数据类型」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Pandas & NumPy Academy 课程的其余内容，请升级到 CoddyKit PRO。 Pandas & NumPy Academy 课程共包含 4 节课。

「分类数据类型」这节课中我会学到什么？

将低基数的字符串列转换为 pandas Categorical，以减少内存使用并加快分组操作。 你通过在浏览器中直接运行的动手代码来练习 Pandas & NumPy Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Pandas & NumPy Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Pandas & NumPy Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 3 节课，共 4 节。

「分类数据类型」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Pandas & NumPy Academy 课中编写并运行代码吗？

能。每节 Pandas & NumPy Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

  1. 检查列数据类型
  2. 使用 astype() 转换类型
  3. 分类数据类型
  4. 正确解析日期
← 返回 Pandas & NumPy Academy