0Pricing
Python Academy · Lesson

Merging, Joining, and Data Cleaning

Combine DataFrames and handle missing values professionally.

Merging, Joining, and Data Cleaning is a free Python Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Python Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

pd.merge

pd.merge(left, right, on="key") joins two DataFrames on a common column — similar to SQL JOIN.

import pandas as pd

users = pd.DataFrame({"id":[1,2,3],"name":["A","B","C"]})
orders = pd.DataFrame({"user_id":[1,1,2],"amount":[100,50,200]})
result = pd.merge(users, orders, left_on="id", right_on="user_id")
print(result)

Join Types

Specify how="inner" (default), "left", "right", or "outer" to control which rows are kept.

import pandas as pd

A = pd.DataFrame({"key":["a","b","c"],"val":[1,2,3]})
B = pd.DataFrame({"key":["b","c","d"],"x":[10,20,30]})

print(pd.merge(A, B, on="key", how="left"))   # all of A
print(pd.merge(A, B, on="key", how="outer"))  # all rows

df.join

df.join(other) joins on the index. Faster and simpler when merging on index rather than a column.

import pandas as pd

df1 = pd.DataFrame({"a":[1,2]}, index=["x","y"])
df2 = pd.DataFrame({"b":[3,4]}, index=["x","y"])
print(df1.join(df2))
# x: a=1 b=3
# y: a=2 b=4

concat

pd.concat([df1, df2]) stacks DataFrames vertically (axis=0) or horizontally (axis=1).

import pandas as pd

df1 = pd.DataFrame({"a":[1,2]})
df2 = pd.DataFrame({"a":[3,4]})
print(pd.concat([df1,df2], ignore_index=True))
# a: 1 2 3 4

# Horizontal:
df3 = pd.DataFrame({"b":[5,6]})
print(pd.concat([df1,df3], axis=1))

Handling Missing Values

Detect NaN with isna()/notna(). Fill with fillna(). Drop with dropna().

import pandas as pd, numpy as np

df = pd.DataFrame({"a":[1,np.nan,3],"b":[np.nan,2,3]})
print(df.isna().sum())      # count nulls per column
df["a"] = df["a"].fillna(0)
df = df.dropna()            # drop rows with any null

fillna Strategies

Fill missing values with a constant, forward-fill (ffill), back-fill (bfill), or the column mean.

import pandas as pd, numpy as np

df = pd.DataFrame({"val":[1, np.nan, np.nan, 4]})
print(df["val"].ffill())   # forward fill: 1 1 1 4
print(df["val"].bfill())   # back fill:    1 4 4 4
print(df["val"].fillna(df["val"].mean()))   # fill with mean

Changing Data Types

Use astype() to convert column types. Use pd.to_datetime() for date parsing and pd.to_numeric with errors="coerce" for safe numeric conversion.

import pandas as pd

df = pd.DataFrame({"age":["25","30","35"],"date":["2024-01-01","2024-06-15","2024-12-31"]})
df["age"] = df["age"].astype(int)
df["date"] = pd.to_datetime(df["date"])
print(df.dtypes)

String Cleaning

Pandas' .str accessor provides vectorised string operations: strip(), lower(), replace(), extract().

import pandas as pd

df = pd.DataFrame({"name":["  Alice  "," bob","CAROL"]})
df["clean"] = df["name"].str.strip().str.title()
print(df["clean"])   # Alice / Bob / Carol

Renaming and Reordering Columns

Rename with df.rename(columns={"old":"new"}). Reorder by indexing with a list.

import pandas as pd

df = pd.DataFrame({"a":[1],"b":[2],"c":[3]})
df = df.rename(columns={"a":"alpha","b":"beta"})
df = df[["c","beta","alpha"]]   # reorder
print(df.columns.tolist())      # ['c', 'beta', 'alpha']

Duplicate Detection

Find duplicates with duplicated() and remove them with drop_duplicates().

import pandas as pd

df = pd.DataFrame({"id":[1,2,1,3],"val":["a","b","a","c"]})
print(df.duplicated())           # [F F T F]
print(df[df.duplicated()])       # show duplicates
clean = df.drop_duplicates(subset=["id"])

apply for Row/Column Transformations

df.apply(func, axis=1) applies a function to each row. axis=0 applies to each column.

import pandas as pd

df = pd.DataFrame({"a":[1,2,3],"b":[4,5,6]})
# New column = row sum:
df["total"] = df.apply(lambda row: row["a"] + row["b"], axis=1)
print(df)

Quick Check

What does df.fillna(method="ffill") do?

Recap

Use pd.merge for SQL-style joins and pd.concat for stacking. Handle NaN with isna(), fillna(), and dropna(). Clean strings with the .str accessor. Convert types with astype() and pd.to_datetime().

Frequently asked questions

Is the “Merging, Joining, and Data Cleaning” lesson free?

Yes — the full text of “Merging, Joining, and Data Cleaning” is free to read here on the web, and the Python Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Python Academy course, upgrade to CoddyKit PRO.

What will I learn in “Merging, Joining, and Data Cleaning”?

Combine DataFrames and handle missing values professionally. You practise Python Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Python Academy?

No prior experience is required. Python Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Merging, Joining, and Data Cleaning” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Python Academy lesson?

Yes. Every Python Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Series and DataFrame Fundamentals
  2. Indexing, Filtering, and Boolean Masks
  3. GroupBy, Aggregation, and Pivot Tables
  4. Merging, Joining, and Data Cleaning
← Back to Python Academy