Loading and Auditing the Sales Dataset
Import a CSV of order records, audit dtypes and missing values, and fix date parsing and negative quantity anomalies.
Loading and Auditing the Sales Dataset is a free Pandas & NumPy Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Pandas & NumPy Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Introducing the Sales Dataset
A typical sales dataset contains columns like order_id, order_date, customer_id, product, category, quantity, unit_price, and region. Before any analysis you must load this file and take a first look at its structure. The goal of this first step is to understand what data you have before writing a single formula.
import pandas as pd
df = pd.read_csv('sales.csv')
print(df.shape) # (rows, columns)
print(df.head())Checking Shape and Column Names
After loading, immediately check df.shape to know the dataset dimensions and df.columns to see all column names. This confirms the file loaded correctly and shows whether any column names have unexpected spaces or capitalisation that need cleaning. A mismatch in expected column count is an early warning sign of a corrupted file.
print('Rows, Cols:', df.shape)
print('Columns:', df.columns.tolist())
print('Index:', df.index[:5].tolist())Inspecting Data Types with dtypes
Use df.dtypes or df.info() to see the inferred data type of every column. Common problems include date columns loaded as object (string) and numeric columns loaded as object because of stray commas or currency symbols. Catching dtype issues early prevents silent errors in arithmetic later.
print(df.dtypes)
# More detail including non-null counts
df.info()Counting Missing Values
Run df.isna().sum() to count NaN values per column. Convert to a percentage with df.isna().mean() * 100 to see what fraction of each column is missing. Columns with more than 30–40 % missing usually need a decision: drop the column, impute, or flag as a separate indicator variable.
missing = df.isna().sum()
missing_pct = df.isna().mean() * 100
print(pd.DataFrame({'count': missing, 'pct': missing_pct}))Detecting Duplicate Rows
Duplicate order records inflate revenue totals and distort any per-customer analysis. Use df.duplicated().sum() to count exact duplicates and df.duplicated(subset=['order_id']).sum() to check for repeated order IDs, which should be unique. Identifying duplicates at load time saves hours of debugging later.
print('Exact duplicates:', df.duplicated().sum())
print('Duplicate order_ids:', df.duplicated(subset=['order_id']).sum())
# Preview the duplicated rows
print(df[df.duplicated(subset=['order_id'], keep=False)].head())Fixing Date Column Parsing
A date column loaded as object must be converted with pd.to_datetime() so you can do date arithmetic. Pass format='%Y-%m-%d' if you know the format, or use infer_datetime_format=True for mixed formats. After conversion, extract year and month with the .dt accessor for time-based grouping.
df['order_date'] = pd.to_datetime(df['order_date'], format='%Y-%m-%d')
df['year'] = df['order_date'].dt.year
df['month'] = df['order_date'].dt.month
print(df[['order_date', 'year', 'month']].head())Spotting Negative Quantities
Negative quantity values typically represent returns or data entry errors. Use boolean indexing to find them: df[df['quantity'] < 0]. Decide whether to treat them as legitimate returns (keep with a flag) or as errors (drop or correct). Always document your decision so the pipeline is reproducible.
negatives = df[df['quantity'] < 0]
print(f'Negative quantity rows: {len(negatives)}')
print(negatives.head())
# Flag returns separately
df['is_return'] = df['quantity'] < 0Checking Numeric Column Ranges
Use df.describe() to see the min, max, mean, and quartiles of every numeric column. A unit_price with a minimum of zero or a negative value is suspicious. A quantity maximum far above any reasonable order size might be a data entry error. Sanity-checking ranges is a fast way to spot anomalies.
print(df[['quantity', 'unit_price']].describe())
# Any price exactly 0?
print('Zero price rows:', (df['unit_price'] == 0).sum())Auditing Categorical Columns
For columns like region and category, use df['region'].value_counts() to see all unique values and their frequencies. Look for typos, inconsistent capitalisation (e.g. 'north' vs 'North'), or unexpected values that indicate upstream data issues. Standardise them before any groupby operation.
print(df['region'].value_counts())
print(df['category'].value_counts())
# Normalise capitalisation
df['region'] = df['region'].str.strip().str.title()Creating an Audit Summary
A structured audit summary DataFrame helps communicate data quality issues to stakeholders. Build one by collecting metrics like row count, column count, missing counts, and duplicate counts into a dictionary and converting it to a DataFrame. This summary becomes the first section of your analysis report and justifies every cleaning step you take.
audit = {
'rows': len(df),
'columns': len(df.columns),
'missing_cells': df.isna().sum().sum(),
'duplicate_rows': df.duplicated().sum(),
'date_range_start': df['order_date'].min(),
'date_range_end': df['order_date'].max()
}
for k, v in audit.items():
print(f'{k}: {v}')Saving a Clean Snapshot
After the initial audit and basic fixes (dtype corrections, duplicate removal, flag columns), save a clean snapshot so downstream steps always start from a consistent baseline. Use df.to_csv('sales_clean.csv', index=False) or, for faster reloading, df.to_parquet('sales_clean.parquet'). Parquet preserves dtypes including datetime, which CSV does not.
# Drop exact duplicates before saving
df = df.drop_duplicates(subset=['order_id'], keep='first')
# Save clean snapshot
df.to_parquet('sales_clean.parquet', index=False)
print('Saved', len(df), 'rows to sales_clean.parquet')Quick Check
Test your understanding of Data Analysis concepts from this lesson.
Lesson Recap
In this lesson you learned: loading a CSV and checking shape/columns, auditing dtypes, missing values, duplicates, and negative quantities, and saving a clean snapshot for downstream steps. Next up we explore revenue calculations and feature engineering on the clean dataset.
Frequently asked questions
Is the “Loading and Auditing the Sales Dataset” lesson free?
Yes — the full text of “Loading and Auditing the Sales Dataset” is free to read here on the web, and the Pandas & NumPy Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Pandas & NumPy Academy course, upgrade to CoddyKit PRO.
What will I learn in “Loading and Auditing the Sales Dataset”?
Import a CSV of order records, audit dtypes and missing values, and fix date parsing and negative quantity anomalies. You practise Pandas & NumPy Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Pandas & NumPy Academy?
No prior experience is required. Pandas & NumPy Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Loading and Auditing the Sales Dataset” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Pandas & NumPy Academy lesson?
Yes. Every Pandas & NumPy Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Loading and Auditing the Sales Dataset
- Revenue Calculations and Feature Engineering
- GroupBy Analysis by Region and Category
- Monthly Trend Visualisation