TF-IDF With scikit-learn
Vectorize a corpus in a few lines.
TF-IDF With scikit-learn is a free NLP Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
No Need to Hand-Code It
You understand the math, so now let scikit-learn do the heavy lifting. Its TfidfVectorizer turns raw documents into a weighted matrix in a few lines.
Import the Vectorizer
Everything lives in the feature_extraction.text module. Import TfidfVectorizer and you are ready to vectorize any list of text strings.
from sklearn.feature_extraction.text import TfidfVectorizerYour Corpus Is a List
A corpus is just a Python list of strings, one per document. Each entry is the full text you want scored and compared.
corpus = [
"the cat sat on the mat",
"the dog chased the cat",
]Fit and Transform
Call fit_transform to learn the vocabulary and compute TF-IDF in one step. It returns a sparse matrix of weighted features.
vec = TfidfVectorizer()
X = vec.fit_transform(corpus)
print(X.shape)What fit Learned
The fit step builds the vocabulary and the IDF values from your corpus. After this the vectorizer knows every term and how rare it is.
Inspect the Vocabulary
You can list the learned feature names to see the columns. get_feature_names_out shows each word in vocabulary order. 🔎
print(vec.get_feature_names_out())The Output Is Sparse
Most documents use only a few words, so the matrix is mostly zeros. scikit-learn stores it as a memory-saving sparse matrix by default.
Peek at Real Numbers
Convert a row to a dense array to actually read the weights. The fillers near zero and topic words stand out clearly.
print(X.toarray()[0].round(3))Tune With Parameters
Handy options let you drop rare or common terms instantly. Set min_df and stop_words to clean the vocabulary as you vectorize.
vec = TfidfVectorizer(stop_words="english", min_df=2)Reuse on New Text
Fit once on training data, then call transform on fresh documents. New text is mapped into the exact same vocabulary and IDF scale.
new_docs = ["a new cat appeared"]
X_new = vec.transform(new_docs)Ready for a Model
This weighted matrix plugs straight into any scikit-learn classifier. TF-IDF features are a strong, fast baseline for real text tasks.
Quick Check
Which method learns the vocabulary and computes the TF-IDF matrix together?
Recap
You imported TfidfVectorizer, fit it on a corpus, inspected the sparse output, and learned to reuse it on new text. The math is now a one-liner. ✅
Frequently asked questions
Is the “TF-IDF With scikit-learn” lesson free?
Yes — the full text of “TF-IDF With scikit-learn” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “TF-IDF With scikit-learn”?
Vectorize a corpus in a few lines. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “TF-IDF With scikit-learn” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.