범주형 데이터 형식
카디널리티가 낮은 문자열 열을 pandas Categorical로 변환해 메모리 사용량을 줄이고 groupby 연산을 빠르게 합니다.
범주형 데이터 형식은(는) CoddyKit의 무료 Pandas & NumPy Academy 강의입니다. 이것은 4개 중 3번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Pandas & NumPy Academy 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Pandas & NumPy Academy 강의에는 총 4개의 강의가 포함되어 있습니다.
이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.
What Is the Categorical Dtype?
The Categorical dtype in Pandas is designed for columns that contain a limited set of discrete values (low cardinality), such as gender, status, region, or product category. Instead of storing the full string for every row, Pandas stores the unique values (called categories) once and uses an integer code per row to reference them. This is similar to how databases use lookup tables or enum types.
import pandas as pd
df = pd.DataFrame({
'region': ['North', 'South', 'East', 'North', 'East', 'South'] * 1000
})
# Without Categorical: object dtype stores every string
print('object dtype memory:', df['region'].memory_usage(deep=True))
# With Categorical: only stores 3 unique values + integer codes
df['region_cat'] = df['region'].astype('category')
print('category dtype memory:', df['region_cat'].memory_usage(deep=True))Creating a Categorical Series
Convert a column to Categorical by calling .astype('category') on any Series. The resulting Categorical Series stores the unique values as .cat.categories and the integer position codes as .cat.codes. You can also create a Categorical directly with pd.Categorical() to specify the categories and their order up front.
import pandas as pd
s = pd.Series(['low', 'high', 'med', 'high', 'low'])
s_cat = s.astype('category')
print(s_cat.cat.categories) # Index(['high', 'low', 'med'], dtype='object')
print(s_cat.cat.codes) # integer codes per row
# 0 1
# 1 0
# 2 2
# 3 0
# 4 1Memory Savings with Categorical
The memory savings from Categorical dtype depend on the cardinality ratio (unique values ÷ total rows). With 5 unique strings in a 1,000,000-row column, the object dtype stores 1 million string pointers (~50+ bytes each) while Categorical stores just 5 strings plus 1 million 8-bit integers. The savings can be 5-20x, turning a 50 MB column into 2-5 MB.
import pandas as pd
import numpy as np
n = 1_000_000
statuses = np.random.choice(['active', 'inactive', 'pending'], size=n)
df = pd.DataFrame({'status': statuses})
object_mem = df['status'].memory_usage(deep=True) / 1e6
df['status'] = df['status'].astype('category')
cat_mem = df['status'].memory_usage(deep=True) / 1e6
print(f'Object dtype: {object_mem:.1f} MB')
print(f'Category dtype: {cat_mem:.1f} MB')
print(f'Reduction: {object_mem/cat_mem:.1f}x')Ordered Categorical for Ranking
Sometimes categories have a natural order — e.g., 'low' < 'medium' < 'high', or T-shirt sizes XS < S < M < L < XL. An ordered Categorical captures this ranking, enabling meaningful comparison operators (<, >=, etc.) and ensuring groupby results are sorted in the correct semantic order rather than alphabetically.
import pandas as pd
df = pd.DataFrame({
'priority': ['high', 'low', 'medium', 'high', 'low']
})
priority_type = pd.CategoricalDtype(
categories=['low', 'medium', 'high'],
ordered=True
)
df['priority'] = df['priority'].astype(priority_type)
# Comparison now works correctly
print(df['priority'] >= 'medium')
# 0 True
# 1 False
# 2 True
# 3 True
# 4 FalseSorting Ordered Categoricals
When you sort an ordered Categorical column, Pandas uses the defined category order rather than alphabetical order. This means 'low' comes before 'medium' before 'high' regardless of how they would sort as strings. This is critical for producing correctly ordered summary tables and charts.
import pandas as pd
df = pd.DataFrame({
'severity': pd.Categorical(
['high', 'low', 'medium', 'low', 'high'],
categories=['low', 'medium', 'high'],
ordered=True
),
'count': [5, 20, 8, 15, 3]
})
# sort_values respects the categorical order
print(df.sort_values('severity')[['severity', 'count']])
# severity count
# 1 low 20
# 3 low 15
# 2 medium 8
# 0 high 5
# 4 high 3GroupBy Speed with Categorical
Pandas can optimise groupby() operations on Categorical columns because it knows all possible groups in advance. This can make groupby 2-5x faster on large DataFrames. Additionally, groupby on Categorical returns all categories — including empty ones — which ensures your summary tables always have a row for every expected group, even if some have zero observations.
import pandas as pd
import numpy as np
np.random.seed(0)
df = pd.DataFrame({
'region': pd.Categorical(
np.random.choice(['North', 'South', 'East', 'West'], 10),
categories=['North', 'South', 'East', 'West']
),
'sales': np.random.randint(100, 1000, 10)
})
# observed=False shows ALL categories even empty ones
print(df.groupby('region', observed=False)['sales'].sum())Adding and Removing Categories
Use the .cat accessor to manage the category list. .cat.add_categories() adds new allowed values (useful before inserting new data), and .cat.remove_unused_categories() drops category labels that have no corresponding rows — handy after filtering. You cannot assign a value that is not in the categories without adding it first.
import pandas as pd
s = pd.Categorical(['a', 'b', 'a'], categories=['a', 'b', 'c'])
s = pd.Series(s)
print(s.cat.categories) # Index(['a', 'b', 'c'], dtype='object')
print(s.value_counts()) # a:2, b:1, c:0
# Remove unused category 'c'
s = s.cat.remove_unused_categories()
print(s.cat.categories) # Index(['a', 'b'], dtype='object')Renaming Category Labels
.cat.rename_categories() lets you relabel categories without changing the underlying codes. This is useful when you want to display friendlier names (e.g., 'M' → 'Male') or fix inconsistent label capitalisation without converting back to object and remapping. The method accepts a list (positional) or a dictionary (targeted).
import pandas as pd
s = pd.Series(pd.Categorical(['M', 'F', 'M', 'F'], categories=['M', 'F']))
# Rename using a dict
s = s.cat.rename_categories({'M': 'Male', 'F': 'Female'})
print(s)
# 0 Male
# 1 Female
# 2 Male
# 3 Female
print(s.cat.categories) # Index(['Male', 'Female'], dtype='object')When NOT to Use Categorical
Categorical dtype is a poor choice when cardinality is high — if almost every row has a unique value (like user IDs, free-text comments, or UUIDs), the category index is nearly as large as the original data, giving no memory saving. A rule of thumb: use Categorical only when the number of unique values is less than roughly 50% of the total rows, and especially when unique values number in the tens or hundreds.
import pandas as pd
import numpy as np
df = pd.DataFrame({'user_id': range(1_000_000)})
# High-cardinality: every value is unique — don't use Categorical
object_mem = df['user_id'].astype(str).memory_usage(deep=True) / 1e6
cat_mem = df['user_id'].astype(str).astype('category').memory_usage(deep=True) / 1e6
print(f'String: {object_mem:.1f} MB')
print(f'Category: {cat_mem:.1f} MB')
# Category is WORSE for high-cardinality data!Categorical in Pivot Tables and Groupby
Using Categorical ensures consistent output shape in groupby and pivot table results. Without Categorical, groups that happen to have zero observations are silently omitted from the result — which can cause misaligned output when comparing across different data slices. With Categorical and observed=False, all groups always appear in the output.
import pandas as pd
df = pd.DataFrame({
'quarter': pd.Categorical(
['Q1', 'Q1', 'Q3'],
categories=['Q1', 'Q2', 'Q3', 'Q4']
),
'revenue': [100, 200, 300]
})
# observed=False includes Q2 and Q4 even though they have no rows
result = df.groupby('quarter', observed=False)['revenue'].sum()
print(result)
# quarter
# Q1 300
# Q2 0
# Q3 300
# Q4 0Categorical and Machine Learning
Many scikit-learn estimators expect numeric input. To convert a Categorical column to numbers for a model, use .cat.codes (label encoding) or pd.get_dummies() (one-hot encoding). One-hot encoding avoids implying an ordinal relationship between categories. For ordered categoricals like priority levels, label encoding (the codes directly) is appropriate and carries the ordering information.
import pandas as pd
df = pd.DataFrame({
'colour': pd.Categorical(['red', 'blue', 'green', 'red'])
})
# One-hot encode for unordered categories
one_hot = pd.get_dummies(df['colour'], prefix='colour')
print(one_hot)
# colour_blue colour_green colour_red
# 0 0 0 1
# 1 1 0 0
# 2 0 1 0
# 3 0 0 1Quick Check
Test your understanding of the Categorical dtype.
Lesson Recap
In this lesson you learned: Categorical dtype stores unique values once and uses integer codes per row, providing large memory savings for low-cardinality columns, ordered Categorical supports meaningful comparisons and correct sorting, and groupby with observed=False includes all categories even empty ones. Avoid Categorical for high-cardinality columns. Next up we parse date strings correctly with pd.to_datetime().
자주 묻는 질문
“범주형 데이터 형식” 강의는 무료인가요?
네 — “범주형 데이터 형식” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Pandas & NumPy Academy 강의 전체를 잠금 해제할 수 있습니다. Pandas & NumPy Academy 강의에는 총 4개의 강의가 포함되어 있습니다.
“범주형 데이터 형식”에서 뭘 배우나요?
카디널리티가 낮은 문자열 열을 pandas Categorical로 변환해 메모리 사용량을 줄이고 groupby 연산을 빠르게 합니다. 브라우저에서 직접 실행하는 실습 코드로 Pandas & NumPy Academy을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.
Pandas & NumPy Academy을(를) 시작하는 데 경험이 필요한가요?
사전 경험은 필요하지 않습니다. CoddyKit의 Pandas & NumPy Academy은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 3번째 강의입니다.
“범주형 데이터 형식” 강의는 얼마나 걸리나요?
대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.
이 Pandas & NumPy Academy 강의에서 코드를 작성하고 실행할 수 있나요?
네. 모든 Pandas & NumPy Academy 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.