Pattern Matching with Regex
Extract substrings and check patterns with .str.extract(), .str.contains(), and .str.match() using regular expressions.
Pattern Matching with Regex is a free Pandas & NumPy Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Pandas & NumPy Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Regex in Pandas?
Regular expressions (regex) are a language for describing string patterns. In Pandas, the .str accessor exposes regex through contains(), match(), extract(), findall(), and replace(). Regex is the right tool when your patterns are too complex for simple contains/startswith checks — for example, validating email formats, extracting phone numbers from free text, or finding all dollar amounts in a notes column.
import pandas as pd
df = pd.DataFrame({
'text': ['Order #12345 placed', 'No order', 'Ref #67890 paid', 'Refund for #11111']
})
# Find rows that contain an order number (# followed by 5 digits)
has_order = df['text'].str.contains(r'#\d{5}', regex=True)
print(df[has_order])
# text
# 0 Order #12345 placed
# 2 Ref #67890 paid
# 3 Refund for #11111.str.contains() with Regex
.str.contains(pattern) with regex=True (the default) returns a boolean Series that is True wherever the pattern matches anywhere in the string. Use anchors like ^ (start) and $ (end) to restrict where the match occurs. Common uses: filter rows where a field matches a format requirement, like a valid date pattern or a properly formatted code.
import pandas as pd
df = pd.DataFrame({
'code': ['US-001', 'UK-042', 'DE-18', 'FR-', 'us-003']
})
# Valid format: two uppercase letters, hyphen, 3 digits
valid = df['code'].str.contains(r'^[A-Z]{2}-\d{3}$', regex=True)
print(df['code'][valid].tolist())
# ['US-001', 'UK-042'].str.match() for Anchored Matching
.str.match(pattern) checks whether each string starts with the pattern (it is anchored at the beginning). This is the difference from contains() which searches anywhere in the string. Use match() when you only care about the prefix of the string, and fullmatch() when the entire string must match the pattern.
import pandas as pd
df = pd.DataFrame({'id': ['ORD001', 'ORD002', 'RET003', 'ORDXYZ']})
# Match rows whose ID starts with 'ORD' followed by digits
orders = df['id'].str.match(r'ORD\d+')
print(df[orders])
# id
# 0 ORD001
# 1 ORD002
# match() is anchored at start, so 'ORDXYZ' (XYZ not digits) is excluded.str.extract() for Capturing Groups
.str.extract(pattern) uses capturing groups () in the regex pattern to pull out specific parts of matching strings. It returns a DataFrame with one column per capturing group. Only the first match in each string is returned. This is perfect for extracting structured data embedded in free text — like numeric values, dates, or IDs.
import pandas as pd
df = pd.DataFrame({
'notes': ['Shipped on 2024-01-15', 'Delivered on 2024-03-20', 'Pending', 'Shipped on 2024-07-04']
})
# Extract the date (YYYY-MM-DD) from the notes column
dates = df['notes'].str.extract(r'(\d{4}-\d{2}-\d{2})')
df['shipped_date'] = dates[0]
print(df[['notes', 'shipped_date']])
# notes shipped_date
# 0 Shipped on 2024-01-15 2024-01-15
# 1 Delivered on 2024-03-20 2024-03-20
# 2 Pending NaN
# 3 Shipped on 2024-07-04 2024-07-04Named Capturing Groups
You can name capturing groups using (?P<name>...) syntax. When you use named groups with str.extract(), the resulting DataFrame automatically uses those names as column headers — making the output self-documenting without needing to rename columns afterward.
import pandas as pd
df = pd.DataFrame({
'entry': ['Alice (25, Engineer)', 'Bob (30, Manager)', 'Carol (28, Analyst)']
})
# Named capturing groups
extracted = df['entry'].str.extract(
r'(?P<name>\w+) \((?P<age>\d+), (?P<role>\w+)\)'
)
print(extracted)
# name age role
# 0 Alice 25 Engineer
# 1 Bob 30 Manager
# 2 Carol 28 Analyst.str.extractall() for Multiple Matches
.str.extractall(pattern) finds all matches of the pattern in each string (not just the first). It returns a DataFrame with a MultiIndex: the outer level is the original row index and the inner level (match) counts the matches per row. This is useful for extracting all numbers, URLs, or keywords from free-text fields.
import pandas as pd
df = pd.DataFrame({
'text': ['Call 555-1234 or 555-5678', 'Contact 800-9000', 'No numbers']
})
# Extract all phone patterns
all_phones = df['text'].str.extractall(r'(\d{3}-\d{4})')
print(all_phones)
# 0
# match
# 0 0 555-1234
# 1 555-5678
# 1 0 800-9000.str.findall() for a List of Matches
.str.findall(pattern) returns a Series of lists, where each list contains all matches found in that row's string. Unlike extractall(), it does not create a MultiIndex — each row gets a list of matches. This is convenient when you want to keep the results in a flat structure or count matches per row.
import pandas as pd
df = pd.DataFrame({
'text': ['Buy 3 items for $10 each', 'Save $5 on 2 products', 'No deal']
})
# Find all dollar amounts
df['prices'] = df['text'].str.findall(r'\$\d+')
df['price_count'] = df['prices'].str.len()
print(df[['text', 'prices', 'price_count']])
# text prices price_count
# 0 Buy 3 items for $10 each [$10] 1
# 1 Save $5 on 2 products [$5] 1
# 2 No deal [] 0Case-Insensitive Matching with re.IGNORECASE
By default, regex in Pandas is case-sensitive. Pass flags=re.IGNORECASE (or re.I) to make the pattern case-insensitive. This avoids having to write alternation patterns like [Pp][Yy][Tt][Hh][Oo][Nn] when you want to match 'python', 'Python', or 'PYTHON' equally.
import pandas as pd
import re
df = pd.DataFrame({
'skill': ['Python', 'JAVA', 'javascript', 'PyThOn developer', 'SQL']
})
# Case-insensitive search for python
mask = df['skill'].str.contains('python', flags=re.IGNORECASE, regex=True)
print(df[mask])
# skill
# 0 Python
# 3 PyThOn developerValidating Formats with Full Match
.str.fullmatch(pattern) (added in Pandas 1.1) returns True only when the entire string matches the pattern. Unlike contains() which matches a substring, fullmatch is equivalent to anchoring with ^...$. This is the safest way to validate format compliance — email addresses, phone numbers, postal codes, or any field with a strict schema.
import pandas as pd
df = pd.DataFrame({
'email': ['alice@example.com', 'bob_at_work', 'carol@test', 'dave@org.co']
})
# Simple email validation: word@word.word
valid_email = df['email'].str.fullmatch(r'[\w.+-]+@[\w-]+\.[\w.]+')
print(df[valid_email])
# email
# 0 alice@example.com
# 3 dave@org.coReplacing with Regex Backreferences
When using str.replace(pattern, repl, regex=True), the replacement string can include backreferences like r'\1' to refer to the content captured in the first group (). This enables powerful reformatting — for example, switching MM/DD/YYYY to YYYY-MM-DD or wrapping a matched word in HTML tags.
import pandas as pd
df = pd.DataFrame({'phone': ['(555) 123-4567', '(800) 555-0199', '(212) 867-5309']})
# Reformat to 555-123-4567
df['phone_clean'] = df['phone'].str.replace(
r'\((\d{3})\) (\d{3})-(\d{4})',
r'\1-\2-\3',
regex=True
)
print(df)
# phone phone_clean
# 0 (555) 123-4567 555-123-4567
# 1 (800) 555-0199 800-555-0199
# 2 (212) 867-5309 212-867-5309Building a Regex-Based Data Extractor
Here is a practical end-to-end example: extracting product SKU and price from a messy notes column using named groups. This type of extraction is common when importing data from legacy systems where multiple fields were concatenated into one text field.
import pandas as pd
df = pd.DataFrame({
'note': [
'SKU:A001 price:$49.99 in stock',
'SKU:B202 price:$12.50 low stock',
'No SKU available'
]
})
extracted = df['note'].str.extract(
r'SKU:(?P<sku>\w+).*price:\$(?P<price>[\d.]+)'
)
print(extracted)
# sku price
# 0 A001 49.99
# 1 B202 12.50
# 2 NaN NaNQuick Check
Test your understanding of pattern matching with regex in Pandas.
Lesson Recap
In this lesson you learned: str.contains(regex) filters rows by pattern, str.extract() captures the first match into a DataFrame column (use named groups for self-documenting output), str.extractall() finds all matches per row with a MultiIndex, and str.fullmatch() validates that the entire string conforms to a pattern. Next up we combine and clean multiple text columns.
Frequently asked questions
Is the “Pattern Matching with Regex” lesson free?
Yes — the full text of “Pattern Matching with Regex” is free to read here on the web, and the Pandas & NumPy Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Pandas & NumPy Academy course, upgrade to CoddyKit PRO.
What will I learn in “Pattern Matching with Regex”?
Extract substrings and check patterns with .str.extract(), .str.contains(), and .str.match() using regular expressions. You practise Pandas & NumPy Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Pandas & NumPy Academy?
No prior experience is required. Pandas & NumPy Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Pattern Matching with Regex” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Pandas & NumPy Academy lesson?
Yes. Every Pandas & NumPy Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The .str Accessor
- Splitting and Replacing Strings
- Pattern Matching with Regex
- Combining and Cleaning Text Columns