Lemmatization: Smarter Base Forms
Reduce words to real dictionary forms.
Lemmatization: Smarter Base Forms is a free NLP Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
A Smarter Reducer
Stemming chops blindly, but lemmatization reduces a word to its real dictionary base form, called the lemma. The result is always a proper word. 📖
Lemma vs Stem
Stemming turns studies into studi, but lemmatization returns the clean lemma study. That tidy, valid output is the big win of lemmatization.
It Uses a Vocabulary
A lemmatizer looks words up in a real vocabulary like WordNet. That lookup is what lets it know better is the base form better, not bett.
The WordNet Lemmatizer
NLTK provides the WordNetLemmatizer. Create one, then call lemmatize on a word to get its base form back.
from nltk.stem import WordNetLemmatizer
lem = WordNetLemmatizer()
print(lem.lemmatize('cars'))Part of Speech Matters
By default the lemmatizer assumes every word is a noun. So the verb running stays running unless you tell it the word is a verb.
print(lem.lemmatize('running'))Pass the Right Tag
Give it the pos argument to fix that. Telling the lemmatizer the word is a verb reduces running all the way down to its base form run.
print(lem.lemmatize('running', pos='v'))Irregular Forms Handled
Because it knows real grammar, the lemmatizer maps tricky words too. The verb went correctly becomes its base form go, something a stemmer cannot do.
print(lem.lemmatize('went', pos='v'))Adjectives and Plurals
It handles other word types as well. With the adjective tag, the lemmatizer turns better into its base form good.
print(lem.lemmatize('better', pos='a'))Lemmatize a Token List
Apply it across a document with a comprehension, just like stemming. Each token comes back as a clean, valid base form.
words = ['dogs', 'boxes', 'wishes']
print([lem.lemmatize(w) for w in words])Slower but Sharper
Lemmatization needs a dictionary and ideally part-of-speech tags, so it runs slower than stemming. You trade speed for cleaner, more accurate results. ⚖️
Which Should You Pick?
Use stemming for fast, large-scale search and lemmatization when readable, accurate base forms matter, like building features for a model.
Quick Check
Let's compare the two reducers.
Recap
You learned lemmatization uses a vocabulary and part-of-speech tags to return real base forms. It is slower than stemming but far cleaner. Great job! 🎉
Frequently asked questions
Is the “Lemmatization: Smarter Base Forms” lesson free?
Yes — the full text of “Lemmatization: Smarter Base Forms” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Lemmatization: Smarter Base Forms”?
Reduce words to real dictionary forms. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Lemmatization: Smarter Base Forms” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Case and Spacing Matter
- Lowercasing and Stripping Whitespace
- Stemming: Chopping to the Root
- Lemmatization: Smarter Base Forms