0Pricing
Learn AI with Python · Lesson

EDA Workflow and Data Profiling

df.info(), df.describe(), missing value audit, data types check, cardinality analysis.

EDA Workflow and Data Profiling is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What Is Exploratory Data Analysis?

Exploratory Data Analysis (EDA) is the first thing you do with any dataset before modeling. Its goal is to understand structure, spot problems, and build intuition.

A disciplined EDA first-look checklist answers: How big is the data? What types are the columns? Where are the missing values? Are there duplicates? How are values distributed?

  • Catch data-quality issues early
  • Decide which features need cleaning
  • Form hypotheses to test later

Loading the Data

Everything starts with loading a DataFrame and taking a quick peek with head() and shape.

shape returns a (rows, columns) tuple so you instantly know the scale you are working with.

import pandas as pd

df = pd.read_csv("customers.csv")
print(df.shape)        # (10000, 8)
print(df.head())       # first 5 rows

df.info() — Types and Non-Null Counts

df.info() is the single most useful first command. It prints each column name, its dtype, and the non-null count.

If a numeric column shows up as object, something is wrong (stray text, commas, currency symbols). If non-null counts differ across columns, you have missing data.

df.info()
# <class pandas.DataFrame>
# RangeIndex: 10000 entries
# age      9800 non-null  float64
# city     10000 non-null object
# income   9500 non-null  float64

df.describe() — Numeric Summary

df.describe() gives count, mean, std, min, quartiles (25/50/75%) and max for every numeric column.

Scan it for red flags: a min of -1 in an age column, a max wildly above the 75th percentile (outliers), or a mean far from the median (skew).

print(df.describe())
# Use include="all" to also summarize categorical columns
print(df.describe(include="all"))

df.isnull().sum() — Counting Missing Values

Chaining isnull() with sum() counts missing values per column. Add another .sum() for the grand total.

Convert to a percentage to judge severity: a column that is 60% missing may be worth dropping, while 2% missing is easy to impute.

print(df.isnull().sum())

missing_pct = df.isnull().sum() / len(df) * 100
print(missing_pct.sort_values(ascending=False))

df.duplicated().sum() — Finding Duplicate Rows

df.duplicated() returns a boolean Series marking rows that are exact copies of an earlier row. Summing it counts how many duplicates exist.

You can scope duplicates to key columns with subset — useful when an id should be unique.

print(df.duplicated().sum())            # exact duplicate rows
print(df.duplicated(subset=["id"]).sum()) # duplicate ids

df = df.drop_duplicates()

value_counts() — Category Frequencies

value_counts() tallies how often each unique value appears in a column. It is the go-to tool for categorical inspection.

Use normalize=True to see proportions instead of raw counts, and dropna=False to include missing values in the tally.

print(df["city"].value_counts())
print(df["city"].value_counts(normalize=True))
print(df["city"].value_counts(dropna=False))

Cardinality Check

Cardinality is the number of distinct values in a column, given by nunique().

  • Low cardinality (a handful of categories) → good for one-hot encoding.
  • High cardinality (thousands of unique strings, e.g. user IDs) → usually not a useful feature as-is.
for col in df.select_dtypes("object").columns:
    print(col, "->", df[col].nunique(), "unique")

Spotting Constant and ID-Like Columns

Two extremes are worth flagging during profiling:

  • Constant columns (nunique() == 1) carry no information — drop them.
  • ID-like columns (nunique() == len(df)) are unique per row and leak nothing useful to most models.
n = len(df)
for col in df.columns:
    u = df[col].nunique()
    if u == 1:
        print(col, "is constant")
    elif u == n:
        print(col, "is ID-like")

dtypes and Memory Usage

Check df.dtypes to confirm columns are stored correctly, and df.memory_usage(deep=True) to find heavy columns.

Downcasting numeric types and converting low-cardinality strings to category can shrink memory dramatically on large datasets.

print(df.dtypes)
print(df.memory_usage(deep=True))

df["city"] = df["city"].astype("category")

A Reusable Profiling Snippet

You can wrap the whole checklist into one helper so every new dataset gets the same first-look treatment.

Run it the moment you load any DataFrame to immediately surface shape, types, missingness, duplicates and cardinality.

def profile(df):
    print("Shape:", df.shape)
    print(df.info())
    print("Missing:\n", df.isnull().sum())
    print("Dupes:", df.duplicated().sum())
    for c in df.select_dtypes("object"):
        print(c, df[c].nunique(), "unique")

profile(df)

Quick Check: Missing Values

You want the percentage of missing values in each column.

Recap: The EDA First-Look Checklist

You now have a repeatable opening routine for any dataset:

  • shape and head() for scale and a preview
  • info() for dtypes and non-null counts
  • describe() for numeric summaries and outlier hints
  • isnull().sum() for missingness
  • duplicated().sum() for duplicate rows
  • value_counts() and nunique() for category frequencies and cardinality

Next we drill into individual columns with univariate analysis.

Frequently asked questions

Is the “EDA Workflow and Data Profiling” lesson free?

Yes — the full text of “EDA Workflow and Data Profiling” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “EDA Workflow and Data Profiling”?

df.info(), df.describe(), missing value audit, data types check, cardinality analysis. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “EDA Workflow and Data Profiling” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. EDA Workflow and Data Profiling
  2. Univariate Analysis
  3. Bivariate and Multivariate Analysis
  4. Feature Distribution and Target Analysis
← Back to Learn AI with Python