Compter avec CountVectorizer

The CountVectorizer from scikit-learn builds your vocabulary and counts words for you in just a few lines. 🚀

First make an instance. With no arguments it uses sensible defaults for tokenizing and lowercasing text.

Calling fit scans your documents and discovers every unique word, building the vocabulary automatically.

Then transform turns each document into its count vector, giving you a numeric matrix of word frequencies.

The shortcut fit_transform does both steps at once on your training text, which is the common workflow.

Inspect the learned words and their indices with get_feature_names_out. These are your matrix columns.

To save memory, the result is a sparse matrix that stores only the non-zero counts, not every zero slot.

Convert to a dense array to actually read the counts. Use toarray for small examples only.

By default it lowercases text and splits on word boundaries, so The and the count as the same token.

You can pass stop_words, min_df, or max_features to prune the vocabulary right inside the vectorizer.

Never refit on test data. Call only transform so new documents use the exact same vocabulary you trained.

Which call learns the vocabulary and counts in one step?

You used CountVectorizer to fit a vocabulary, transform text into counts, inspect features, and reuse it on new data. 🎉