0Pricing
Learn AI with Python · Lesson

Text Cleaning for AI with Regex

Removing HTML tags, punctuation, URLs, emails — building text preprocessing functions.

Text Cleaning for AI with Regex is a free Learn AI with Python lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Clean Text for AI?

Raw text from the web is full of noise: HTML tags, URLs, emails, extra whitespace, special characters. Feeding that to a model wastes capacity and hurts quality.

Regex is the workhorse of text preprocessing. We will build a cleaning pipeline step by step.

A Messy Sample

Here is the kind of string you might scrape. Each cleaning step targets one type of noise.

raw = "<p>Contact us at info@shop.com or visit https://shop.com NOW!!!   Thanks</p>"

Stripping HTML Tags

HTML tags look like <tag>. The pattern <[^>]+> matches a <, any non-> characters, then >.

Replace them with an empty string. (For complex HTML, prefer a parser like BeautifulSoup, but regex is fine for quick cleanup.)

import re

text = re.sub("<[^>]+>", "", raw)
print(text)   # "Contact us at info@shop.com or visit https://shop.com NOW!!!   Thanks"

Removing URLs

URLs start with http:// or https:// followed by non-space characters. Match and strip them.

import re

text = re.sub("https?://\S+", "", text)
print(text)   # URL removed

Removing Email Addresses

A simple email pattern: word characters, an @, a domain, a dot, and a top-level domain.

import re

text = re.sub("\S+@\S+\.\S+", "", text)
print(text)   # email removed

Masking Instead of Removing

For privacy datasets you often mask sensitive data rather than delete it, preserving sentence structure.

import re

masked = re.sub("\S+@\S+\.\S+", "[EMAIL]", "reach me at a@b.com please")
print(masked)   # "reach me at [EMAIL] please"

Removing Special Characters

Keep letters, digits, and spaces; drop punctuation and symbols with a negated set. Decide per task whether to keep some punctuation.

import re

text = re.sub("[^A-Za-z0-9 ]", "", "Hello!! @world #2026")
print(text)   # "Hello world 2026"

Normalizing Whitespace

Cleaning leaves behind double spaces and stray newlines. Collapse any run of whitespace to a single space with \s+, then strip() the ends.

import re

text = re.sub("\s+", " ", "Hello    world\n\n  again").strip()
print(text)   # "Hello world again"

Lowercasing and Order Matters

Lowercasing is common for consistency, but apply it carefully. Also, order matters: strip HTML before removing special characters, and normalize whitespace last so earlier removals do not leave gaps.

text = text.lower()
# Pipeline order: tags -> urls -> emails -> specials -> whitespace -> lowercase

Building a Cleaning Pipeline

Wrap the steps into one function so every document gets the same treatment. Pre-compile patterns for speed when processing many documents.

import re

def clean_text(s):
    s = re.sub("<[^>]+>", " ", s)          # html tags
    s = re.sub("https?://\S+", " ", s)     # urls
    s = re.sub("\S+@\S+\.\S+", " ", s)    # emails
    s = re.sub("[^A-Za-z0-9 ]", " ", s)     # special chars
    s = re.sub("\s+", " ", s).strip()       # whitespace
    return s.lower()

print(clean_text(raw))

Applying the Pipeline to a DataFrame

Run the cleaner over a whole text column with apply, producing a model-ready column.

import pandas as pd

df["clean"] = df["text"].apply(clean_text)
print(df[["text", "clean"]].head())

Quick Check: Whitespace Normalization

After cleaning, your text has runs of multiple spaces and newlines you want collapsed to single spaces.

Recap: Text Cleaning with Regex

You built a real preprocessing pipeline:

  • Strip HTML with <[^>]+>
  • Remove URLs (https?://\S+) and emails
  • Mask sensitive data with placeholders like [EMAIL]
  • Drop special characters with a negated set
  • Normalize whitespace with \s+ + strip()
  • Wrap it in a function and apply it to a column

That completes regex for text AI. Next: databases for AI projects.

Frequently asked questions

Is the “Text Cleaning for AI with Regex” lesson free?

Yes — the full text of “Text Cleaning for AI with Regex” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “Text Cleaning for AI with Regex”?

Removing HTML tags, punctuation, URLs, emails — building text preprocessing functions. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Text Cleaning for AI with Regex” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Regex Patterns and Character Classes
  2. re Module: search, match, findall, sub
  3. Capturing Groups and Named Groups
  4. Text Cleaning for AI with Regex
← Back to Learn AI with Python