보고서에서 결과 요약하기

구조화된 EDA 보고서가 중요한 이유

EDA 보고서의 여섯 가지 구성

데이터셋 개요 섹션 작성

import pandas as pd
import seaborn as sns

df = sns.load_dataset('titanic')

print('=== DATASET OVERVIEW ===')
print(f'Shape: {df.shape[0]:,} rows x {df.shape[1]} columns')
print(f'Columns: {", ".join(df.columns)}')
print(f'Numeric: {df.select_dtypes("number").shape[1]} columns')
print(f'Categorical: {df.select_dtypes("object").shape[1]} columns')
print(f'Boolean: {df.select_dtypes("bool").shape[1]} columns')
print(f'Total missing cells: {df.isna().sum().sum():,}')
print(f'Duplicate rows: {df.duplicated().sum()}')

데이터 품질 섹션 작성

import pandas as pd
import seaborn as sns

df = sns.load_dataset('titanic')

# Auto-generate data quality table
missing_pct = (df.isna().sum() / len(df) * 100).round(1)
quality = missing_pct[missing_pct > 0].to_frame(name='missing_pct')
quality['severity'] = quality['missing_pct'].apply(
    lambda x: 'High' if x > 30 else ('Medium' if x > 10 else 'Low')
)
quality['action'] = quality['missing_pct'].apply(
    lambda x: 'Drop column' if x > 50 else 'Impute or flag'
)
print(quality.to_string())

다중 패널 요약 그림 생성

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

df = sns.load_dataset('titanic')

fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# 1. Fare distribution
sns.histplot(df['fare'], kde=True, bins=30, ax=axes[0][0])
axes[0][0].set_title('Fare is right-skewed (skew=4.1)')

# 2. Survival by class
survival = df.groupby('class')['survived'].mean().reset_index()
sns.barplot(data=survival, x='class',
            y='survived', order=['First', 'Second', 'Third'], ax=axes[0][1])
axes[0][1].set_title('Survival Rate Drops by Class')

# 3. Age distribution
sns.histplot(df['age'].dropna(), kde=True, bins=25, ax=axes[1][0], color='coral')
axes[1][0].set_title('Age: Near-Normal, Missing 20%')

# 4. Embarkation counts
sns.countplot(data=df, x='embarked', ax=axes[1][1], palette='Set2')
axes[1][1].set_title('Most Boarded at Southampton')

plt.tight_layout()
plt.savefig('/tmp/eda_summary.png', dpi=150, bbox_inches='tight')
plt.show()
print('Saved: /tmp/eda_summary.png')

발견 사항을 서술형 텍스트로 작성

프로그래밍 방식으로 마크다운 보고서 생성

import pandas as pd
import seaborn as sns

df = sns.load_dataset('titanic')

# Build a simple markdown report
lines = [
    '# EDA Summary Report: Titanic Dataset',
    '',
    '## Overview',
    f'- **Rows:** {len(df):,}',
    f'- **Columns:** {df.shape[1]}',
    f'- **Total Missing Cells:** {df.isna().sum().sum():,}',
    '',
    '## Key Findings',
    '- First-class passengers had a 63% survival rate vs 24% for third class.',
    '- Fare is extremely right-skewed (skew=4.1); use log transform before modelling.',
    '- Age is missing in 20% of rows — median imputation recommended.',
    '',
    '## Recommended Next Steps',
    '1. Impute age with median grouped by class.',
    '2. Drop the cabin column (77% missing).',
    '3. Log-transform fare before any regression modelling.',
]

with open('/tmp/eda_report.md', 'w') as f:
    f.write('\n'.join(lines))

print('Report written to /tmp/eda_report.md')
print('\n'.join(lines[:10]))

Jupyter로 HTML로 내보내기

# To export a Jupyter notebook to HTML:
# jupyter nbconvert --to html eda_notebook.ipynb

# To parameterise and run with papermill:
# pip install papermill
# papermill eda_template.ipynb eda_2024.ipynb -p date '2024-01-01' -p min_rows 1000

# The output notebook can then be exported:
# jupyter nbconvert --to html eda_2024.ipynb
print('Jupyter nbconvert and papermill automate report generation.')

실행 가능한 다음 단계 구조화

신속한 보고를 위한 pandas-profiling 사용

# pip install ydata-profiling
# from ydata_profiling import ProfileReport
# import seaborn as sns

# df = sns.load_dataset('titanic')
# profile = ProfileReport(df, title='Titanic EDA')
# profile.to_file('titanic_eda.html')  # interactive HTML
# profile.to_notebook_iframe()          # inline in Jupyter

# Key sections in the generated report:
# - Overview: shape, missing, duplicates
# - Variables: per-column statistics and plots  
# - Correlations: Pearson, Spearman, Cramers V
# - Missing values: heatmap and dendrogram
# - Duplicates: full list of duplicate rows
print('ydata-profiling generates full EDA reports with one function call.')

피해야 할 EDA 보고서 안티 패턴

간단한 확인

단원 요약