Pandas & NumPy Academy · レッスン

売上データセットの読み込みと監査

注文レコードのCSVを読み込み、dtypesと欠損値を監査し、日付の解析や数量が負になる異常を修正します。

「売上データセットの読み込みと監査」はCoddyKit上の無料Pandas & NumPy Academyレッスンです。これはレッスン1/4です。下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。これはPandas & NumPy Academy学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Pandas & NumPy Academyコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

Introducing the Sales Dataset

A typical sales dataset contains columns like order_id, order_date, customer_id, product, category, quantity, unit_price, and region. Before any analysis you must load this file and take a first look at its structure. The goal of this first step is to understand what data you have before writing a single formula.

import pandas as pd

df = pd.read_csv('sales.csv')
print(df.shape)       # (rows, columns)
print(df.head())

Checking Shape and Column Names

After loading, immediately check df.shape to know the dataset dimensions and df.columns to see all column names. This confirms the file loaded correctly and shows whether any column names have unexpected spaces or capitalisation that need cleaning. A mismatch in expected column count is an early warning sign of a corrupted file.

print('Rows, Cols:', df.shape)
print('Columns:', df.columns.tolist())
print('Index:', df.index[:5].tolist())

Inspecting Data Types with dtypes

Use df.dtypes or df.info() to see the inferred data type of every column. Common problems include date columns loaded as object (string) and numeric columns loaded as object because of stray commas or currency symbols. Catching dtype issues early prevents silent errors in arithmetic later.

print(df.dtypes)

# More detail including non-null counts
df.info()

Counting Missing Values

Run df.isna().sum() to count NaN values per column. Convert to a percentage with df.isna().mean() * 100 to see what fraction of each column is missing. Columns with more than 30–40 % missing usually need a decision: drop the column, impute, or flag as a separate indicator variable.

missing = df.isna().sum()
missing_pct = df.isna().mean() * 100

print(pd.DataFrame({'count': missing, 'pct': missing_pct}))

Detecting Duplicate Rows

Duplicate order records inflate revenue totals and distort any per-customer analysis. Use df.duplicated().sum() to count exact duplicates and df.duplicated(subset=['order_id']).sum() to check for repeated order IDs, which should be unique. Identifying duplicates at load time saves hours of debugging later.

print('Exact duplicates:', df.duplicated().sum())
print('Duplicate order_ids:', df.duplicated(subset=['order_id']).sum())

# Preview the duplicated rows
print(df[df.duplicated(subset=['order_id'], keep=False)].head())

Fixing Date Column Parsing

A date column loaded as object must be converted with pd.to_datetime() so you can do date arithmetic. Pass format='%Y-%m-%d' if you know the format, or use infer_datetime_format=True for mixed formats. After conversion, extract year and month with the .dt accessor for time-based grouping.

df['order_date'] = pd.to_datetime(df['order_date'], format='%Y-%m-%d')

df['year'] = df['order_date'].dt.year
df['month'] = df['order_date'].dt.month

print(df[['order_date', 'year', 'month']].head())

Spotting Negative Quantities

Negative quantity values typically represent returns or data entry errors. Use boolean indexing to find them: df[df['quantity'] < 0]. Decide whether to treat them as legitimate returns (keep with a flag) or as errors (drop or correct). Always document your decision so the pipeline is reproducible.

negatives = df[df['quantity'] < 0]
print(f'Negative quantity rows: {len(negatives)}')
print(negatives.head())

# Flag returns separately
df['is_return'] = df['quantity'] < 0

Checking Numeric Column Ranges

Use df.describe() to see the min, max, mean, and quartiles of every numeric column. A unit_price with a minimum of zero or a negative value is suspicious. A quantity maximum far above any reasonable order size might be a data entry error. Sanity-checking ranges is a fast way to spot anomalies.

print(df[['quantity', 'unit_price']].describe())

# Any price exactly 0?
print('Zero price rows:', (df['unit_price'] == 0).sum())

Auditing Categorical Columns

For columns like region and category, use df['region'].value_counts() to see all unique values and their frequencies. Look for typos, inconsistent capitalisation (e.g. 'north' vs 'North'), or unexpected values that indicate upstream data issues. Standardise them before any groupby operation.

print(df['region'].value_counts())
print(df['category'].value_counts())

# Normalise capitalisation
df['region'] = df['region'].str.strip().str.title()

Creating an Audit Summary

A structured audit summary DataFrame helps communicate data quality issues to stakeholders. Build one by collecting metrics like row count, column count, missing counts, and duplicate counts into a dictionary and converting it to a DataFrame. This summary becomes the first section of your analysis report and justifies every cleaning step you take.

audit = {
    'rows': len(df),
    'columns': len(df.columns),
    'missing_cells': df.isna().sum().sum(),
    'duplicate_rows': df.duplicated().sum(),
    'date_range_start': df['order_date'].min(),
    'date_range_end': df['order_date'].max()
}

for k, v in audit.items():
    print(f'{k}: {v}')

Saving a Clean Snapshot

After the initial audit and basic fixes (dtype corrections, duplicate removal, flag columns), save a clean snapshot so downstream steps always start from a consistent baseline. Use df.to_csv('sales_clean.csv', index=False) or, for faster reloading, df.to_parquet('sales_clean.parquet'). Parquet preserves dtypes including datetime, which CSV does not.

# Drop exact duplicates before saving
df = df.drop_duplicates(subset=['order_id'], keep='first')

# Save clean snapshot
df.to_parquet('sales_clean.parquet', index=False)
print('Saved', len(df), 'rows to sales_clean.parquet')

Quick Check

Test your understanding of Data Analysis concepts from this lesson.

Lesson Recap

In this lesson you learned: loading a CSV and checking shape/columns, auditing dtypes, missing values, duplicates, and negative quantities, and saving a clean snapshot for downstream steps. Next up we explore revenue calculations and feature engineering on the clean dataset.

よくある質問

「売上データセットの読み込みと監査」レッスンは無料ですか？

はい。「売上データセットの読み込みと監査」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Pandas & NumPy Academyコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Pandas & NumPy Academyコースには全4レッスンが含まれています。

「売上データセットの読み込みと監査」で何を学びますか？

注文レコードのCSVを読み込み、dtypesと欠損値を監査し、日付の解析や数量が負になる異常を修正します。ブラウザで直接実行するハンズオンコードでPandas & NumPy Academyを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Pandas & NumPy Academyを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのPandas & NumPy Academyは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。これはレッスン1/4です。

「売上データセットの読み込みと監査」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このPandas & NumPy Academyレッスンでコードを書いて実行できますか？

はい。すべてのPandas & NumPy Academyレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

← Pandas & NumPy Academyに戻る